In his article, “What Agile QA Really Does: Testing Requirements,” Steve Berczuk argues that teams transitioning to Agile have a hard time knowing what to do with their QA teams because the focus is on developer testing of code.

Although QA teams in Agile should not become the sole testing source, they can assist in testing things that developers may not be completely able to test. In addition to catching coding errors that slipped through, a quick QA team may be able to find unexpected paths through the application and identify problems on these paths. Steve calls this exploratory testing.

In Steve’s words:

If you have a QA team, and they are doing exploratory testing, they are really testing:

Requirements: Finding interactions between features and components that were not defined or understood when coding and developer testing began. 

Developer Tests: Identifying where developer tests were not as good as they could have been (and how they could be better). This is a good use of automated “integration” tests.

Systems tests that might be hard to test in a developer context. This might be the one set of tests that are the primary domain of QA. Read More>>

So what are your thoughts? How can a QA team provide value on an Agile project? We look forward to talking about this with you on our blog.

Our blog focuses on QA and testing and related topics. Did you know Computer Aid has a robust QA and testing solution available through our Delaware Solution Center?

Called Vericenter, this offering allows us to offer enterprise-wide testing, quality assurance, and complex integration services. But why is Vericenter so different?

Vericenter delivers high-quality and cost-effective quality assurance solution to support testing needs enable our clients to focus on strategic business initiatives through:

• Repeatable QA Process and Artifact Improvements
• Leveraged Variable QA Staffing Team (Onshore / Offshore);
• Detailed Dashboard and Metrics
• Minimize costs by staffing flexible resources to address project needs
• Root Cause Analysis for Cost and Defect Savings

As a result, our clients experience reduced costs (Vericenter delivers a 25% savings in the SDLC), increased effectiveness (through process improvements, automation, and metrics), and reduced risks (mitigation of system down-time and application errors through structured testing practices).

Take a look around for yourself. Let us know what you think about our work and services. By all means, contact us if you think you have a project that would be mutually beneficial.

I just received a panicky call from my Father-In-Law.  He is a retired teacher and his state run website “accidently” put up every teachers: Name, Social Security Number and last known address on a public website.    He was incensed… “What should I do?”, “Didn’t anybody check it?”…..”You’re a QA person…How could this happen?”

Obviously, this is a huge mistake and certainly being in the Quality Assurance business this creates “real-world” discussion.  I can not imagine being part of the discussion and root cause analysis of the parties responsible for making this huge “quality assurance Defect”.  Notice I said, Quality Assurance Defect.

The organization certainly did not follow a defined processes (or that process was flawed.) where there were checks through the release of this information on a website.  In IT terms, we call them gates (exit and entry criteria) to allow this go to a production environment. 

What is the cost of this defect?  No one knows at this point (but it’s very high!!!) .……and  I will say….when I get into a discussion regarding the value of Quality Assurance for IT Projects………..it’s very valuable to have real life examples!!!

Is your “QA process” ready for Agile?

The growing popularity of utilizing the Agile Development methodology to develop systems has changed the way software is tested.

However, you also have to ask yourself the question? Are you ready from a QA perspective?

Let’s think about QA as it relates to Agile the questions you need to answer for yourself. If you say ‘no’ to any of these questions then you do really have to consider whether your ready.

• Do you have a ‘automatic’ regression test that can be run on a night basis?
• Is your build process automated and then tested against?
• Have you developed an “integrated” testing team that can lead and test successfully during a sprint?
• Have you looked at your current QA Tools and will they work in an Agile methodology?
• Are your ‘QA artifacts” right weighted for Agile?
• What important QA Metrics are important and operational for successful Agile sprints?
• Is your team makeup right for Agile?

These are just some of the things to consider. If you said “NO” to only few of these questions then you probably will not be successful.

In today’s world there are many systems that are being configured to adapt to ever changing business conditions. The solutions include the use of SOA architecture and Commercial, off-the-shelf (COTS) software where companies are configuring and implementing appropriately.  This creates quite a challenging world to properly validate the quality of an implemented system.  This creates a different approach to testing. 

The most important point is responsibility and accountability for the companies that develop these systems.  The approach is simple but effective.  Once your company decides to go with a vendor, it is critical to set your organization up for success. 

1. Understand the vendors release schedule and make sure it is integrated into your organization.
2. Meet with the vendor and document their QA Process and testing approach (including deliverables)
3. Require the vendor to deliver all testing artifacts with their product releases.

Following these key approaches will help your organization improve quality while delivering value to your organization.

First – I am a big fan of open source tools.  This is a great way to build real solutions in a collaborative way.  However (and that’s a big however), don’t put all your “eggs in one basket”.  Meaning, don’t count on open source tools to address all of your organizations needs for Test Management, Functional Automation and Performance and Scalability testing.

Let’s start by what they these tools are called -   “Open Source Testing Tools”.  This means that you as an organization are able to use them with little or no procurement involvement.  It is important to understand that you as an organization are obligated to not only be an active participant of ideas and collaboration but frankly the maintenance and support of the tool itself (particularly if you bring into your own organization).

Here are a few tools out there that you may what to know about.  Free to offer your opinion on these open source tools.

Functional Testing:

Canoo WebTest

• Development-focused solution for web applications producing HTML (JSP, PGP, etc)
• Scripts written in java or xml
• Provides detailed reporting

Selenium

• Flexible solution for web applications with strong support for JavaScript but limited reporting
• Record/playback or written scripts in “Selenese”
• Scalable for larger testing efforts

Performance Testing:

Apache JMeter

• Pure Java desktop application designed to load test functional behavior and measure performance
• Scripts written in Java
• Supports web, SOAP, LDAP, Database, Mail, etc

OpenSTA

• Windows-based application for performing and measuring scripted heavy load tests
• Record/playback or written scripts in SCL
• Supports HTTP/HTTPS; only manual correlation

In the article by Eric Mumford titled “How is QA different for Waterfall, Agile, and XP?”, the author looks at each of the methodologies and expresses his opinion on how QA works. I have included excerpts of his opinions below. We encourage you to read the full article.

Waterfall: “The QA cycle involves authoring manual tests to match and cover the product requirements, authoring automated scripts to test the product features, updating regression scripts to test the defects that were found in past builds, and to run performance analysis on the build.  … It [sic]  requires larger toolsets to track test cases, testing results, performance testing, and automated scripting.  Waterfall … requires a heavy investment in tools to get the job done because of the copious amount of work that needs to be tracked.”

Agile: “QA focuses on writing automated scripts, usually using an open source test framework, against the software product as the features are being built.  QA prepares a performance test and adjusts the navigation scripts and virtual user balance ratio as necessary based on their estimation of the features in that iteration. When the software is delivered, QA often has only a few days to complete testing rather than weeks or months.”

XP (Extreme Programming): “Only the most technical QA Managers and QA staff members will survive in XP.  QA staff must have a working knowledge of shell script, Ruby/Python, and the deployment process, as well as SEO and ad-ops for web software.  QA will usually work entirely open-source in these environments, implementing tools in a “fast and loose” manner as needed.  Changing direction completely is common.  QA must write the minimal set of functions required to provide an automated test bed, code coverage metrics, manual cross-browser testing, and performance analysis and be able to do so in hours rather than days.”

What are your experiences? Have you been locked into a particular SDLC, and how has the QA used matched or deviated from Eric’s perceptions? Leave your thoughts and comments on our blog.

If you comb the live webinar list (unfortunately at this time it is not searchable), you will find the following QA topics that may be of interest to you:

• Practical QA Testing in a Configurable World
• Setting Up a Testing Center of Excellence: Making it Work
• Science-based Test Planning – Fewer Tests, Better Coverage
• Testing Automation: Keys to Success

And when you can’t attend, we record the webinars and provide them for free viewing for the first week after the webinar has aired. After that time, you can purchase access to the webinar for a nominal fee.

We also keep recordings of older QA webinars in our webinar archives. There is a search feature that allows you to search either by category or by speaker. Although QA is not a search option, you can select Testing as the category, and the archives displays a list of 15+ webinars. Some of the more interesting ones include:

• Book Discussion: Foundations of Software Testing: ISTQB Certification
• Proactive Testing Puts Agile Test-Driven (and Other) Development on Steroids
• Short-Cycle Development during Test: Case Study for a Legacy Enterprise IT System
• The International Software Testing Qualification Board Advanced Syllabus

Look over the full list of webinars and maybe purchase some. Let us know what you’ve learned, what you’d like to see us offer, and what we can improve.

In his article, “Automation Selection Criteria – Picking the “Right” Candidates,” Bob Galen makes the following two assumptions:

One of the basic assumptions I’m making for this article is that you’ve been creating test cases and manually testing as an organization for a while. That is, you’ve built up some sort of repository of manual test cases that are potential automation “candidates”. Given that you can’t automate everything at once, the question of where to start and how to properly orchestrate your efforts over time becomes a challenge. Read More>>

I’m also assuming that you do not have infinite resources nor time to produce visible results. That is, you have other testing responsibilities besides the automation, for example testing and releasing your products. So prioritization and establishing a work balance becomes a challenge as well. Read More>>

Galen then explores the behavioral patterns that impede testing automation, explains the common strategies that are succesful for testing automation, and how to change your selection criteria when your circumstances change.

This article is chock-full of a lot of thought and detail that we encourage you to read for yourself. Then we invite you to come back to our blog and share your experiences with testing automation—whether you’ve gotten mired down in impediments or you’ve tried one of Bob’s strategies for success successfully or how you accomplished a change in selection criteria.

In Galen’s own words, his purpose for writing the article is:

… to help you “connect” your automation efforts to traditional SDLC activities. While some test teams are getting better at it, I still see far too many clients that manage their automation outside of good software development practices. I’d like to see that trend change much more aggressively. Read More>>

Bob’s four key points during the article include the following:

1) Establish the major drivers for creating an Automation SDLC

2) Explore a few of the key success criteria behind a solid Automation SDLC effort

3) Review Automation SDLC extensions from your own product SDLC

4) Finally, consider how to integrate automation correctly with your mainline development efforts Read More>>

He goes into a great deal of depth on each of his key points…too much depth for an online blog. We encourage you to go and read the article and then come back to our blog ready to discuss your views on how to best develop an Automation Development Lifecycle.