I just received a panicky call from my Father-In-Law.  He is a retired teacher and his state run website “accidently” put up every teachers: Name, Social Security Number and last known address on a public website.    He was incensed… “What should I do?”, “Didn’t anybody check it?”…..”You’re a QA person…How could this happen?”

Obviously, this is a huge mistake and certainly being in the Quality Assurance business this creates “real-world” discussion.  I can not imagine being part of the discussion and root cause analysis of the parties responsible for making this huge “quality assurance Defect”.  Notice I said, Quality Assurance Defect.

The organization certainly did not follow a defined processes (or that process was flawed.) where there were checks through the release of this information on a website.  In IT terms, we call them gates (exit and entry criteria) to allow this go to a production environment. 

What is the cost of this defect?  No one knows at this point (but it’s very high!!!) .……and  I will say….when I get into a discussion regarding the value of Quality Assurance for IT Projects………..it’s very valuable to have real life examples!!!

There are lots of Software Development Lifecycles (SDLCs) out there. The selected SDLC affects everything from requirements gathering to QA and testing. So just how does the selected SDLC affect what you do for QA?

In the article by Eric Mumford titled “How is QA different for Waterfall, Agile, and XP?”, the author looks at each of the methodologies and expresses his opinion on how QA works. I have included excerpts of his opinions below. We encourage you to read the full article.

Waterfall: “The QA cycle involves authoring manual tests to match and cover the product requirements, authoring automated scripts to test the product features, updating regression scripts to test the defects that were found in past builds, and to run performance analysis on the build.  … It [sic]  requires larger toolsets to track test cases, testing results, performance testing, and automated scripting.  Waterfall … requires a heavy investment in tools to get the job done because of the copious amount of work that needs to be tracked.”

Agile: “QA focuses on writing automated scripts, usually using an open source test framework, against the software product as the features are being built.  QA prepares a performance test and adjusts the navigation scripts and virtual user balance ratio as necessary based on their estimation of the features in that iteration. When the software is delivered, QA often has only a few days to complete testing rather than weeks or months.”

XP (Extreme Programming): “Only the most technical QA Managers and QA staff members will survive in XP.  QA staff must have a working knowledge of shell script, Ruby/Python, and the deployment process, as well as SEO and ad-ops for web software.  QA will usually work entirely open-source in these environments, implementing tools in a “fast and loose” manner as needed.  Changing direction completely is common.  QA must write the minimal set of functions required to provide an automated test bed, code coverage metrics, manual cross-browser testing, and performance analysis and be able to do so in hours rather than days.”

What are your experiences? Have you been locked into a particular SDLC, and how has the QA used matched or deviated from Eric’s perceptions? Leave your thoughts and comments on our blog.

In our last blog entry, we shared with you the steps for developing a compelling case for test automation. That leaves us with the question – How do you develop an Automation Development Lifecycle? Bob Galen proposes the answer in his article, “Establishing Your Automation Development Lifecycle”.

In Galen’s own words, his purpose for writing the article is:

… to help you “connect” your automation efforts to traditional SDLC activities. While some test teams are getting better at it, I still see far too many clients that manage their automation outside of good software development practices. I’d like to see that trend change much more aggressively. Read More>>

Bob’s four key points during the article include the following:

1) Establish the major drivers for creating an Automation SDLC

2) Explore a few of the key success criteria behind a solid Automation SDLC effort

3) Review Automation SDLC extensions from your own product SDLC

4) Finally, consider how to integrate automation correctly with your mainline development efforts Read More>>

He goes into a great deal of depth on each of his key points…too much depth for an online blog. We encourage you to go and read the article and then come back to our blog ready to discuss your views on how to best develop an Automation Development Lifecycle.

In today’s world there are many systems that are being configured to adapt to ever changing business conditions. The solutions include the use of SOA architecture and Commercial, off-the-shelf (COTS) software where companies are configuring and implementing appropriately.  This creates quite a challenging world to properly validate the quality of an implemented system.  This creates a different approach to testing.

The most important point is responsibility and accountability for the companies that develop these systems.  The approach is simple but effective.  Once your company decides to go with a vendor, it is critical to set your organization up for success.

1. Understand the vendors release schedule and make sure it is integrated into your organization.
2. Meet with the vendor and document their QA Process and testing approach (including deliverables)
3. Require the vendor to deliver all testing artifacts with their product releases.

Following these key approaches will help your organization improve quality while delivering value to your organization.