In software development projects it is a consistent issue.  One could argue that it will never change.  I am not hear to tell you how to fix that point.  I am hear to tell you to plan for it.  The word is process with a pragmatic side to it. 

 The key is the QA Leads need to take control of the situation.  Requirements can change in the middle of Test Preparation (Test Plan, Test Scenario / Test Script development) and even Test Execution.

 It is important that we communicate out to the project teams in a professional manager.  This includes the state and rework.    This is simple proactive communication.  

It reminds me of the a famous Movie line from ‘Good Will Hunting’.  

 Quote:  “It’s not your fault.  It’s not your fault. It’s not your fault.”

 Your ‘fault’ is if you don’t communicate and properly report out status.  You need to do it quickly and effectively.

I just received a panicky call from my Father-In-Law.  He is a retired teacher and his state run website “accidently” put up every teachers: Name, Social Security Number and last known address on a public website.    He was incensed… “What should I do?”, “Didn’t anybody check it?”…..”You’re a QA person…How could this happen?”

Obviously, this is a huge mistake and certainly being in the Quality Assurance business this creates “real-world” discussion.  I can not imagine being part of the discussion and root cause analysis of the parties responsible for making this huge “quality assurance Defect”.  Notice I said, Quality Assurance Defect.

The organization certainly did not follow a defined processes (or that process was flawed.) where there were checks through the release of this information on a website.  In IT terms, we call them gates (exit and entry criteria) to allow this go to a production environment. 

What is the cost of this defect?  No one knows at this point (but it’s very high!!!) .……and  I will say….when I get into a discussion regarding the value of Quality Assurance for IT Projects………..it’s very valuable to have real life examples!!!

The IT Metrics and Productivity Institute (ITMPI) is a sister organization to Computer Aid. ITMPI’s main goal is to promote best practices in the areas of Software Process, Software Metrics, Software Estimation, and IT Governance. One of the ways it does this is by offering live webinars and maintaining an archive of previously recorded webinars.

If you comb the live webinar list (unfortunately at this time it is not searchable), you will find the following QA topics that may be of interest to you:

• Practical QA Testing in a Configurable World
• Setting Up a Testing Center of Excellence: Making it Work
• Science-based Test Planning – Fewer Tests, Better Coverage
• Testing Automation: Keys to Success

And when you can’t attend, we record the webinars and provide them for free viewing for the first week after the webinar has aired. After that time, you can purchase access to the webinar for a nominal fee.

We also keep recordings of older QA webinars in our webinar archives. There is a search feature that allows you to search either by category or by speaker. Although QA is not a search option, you can select Testing as the category, and the archives displays a list of 15+ webinars. Some of the more interesting ones include:

• Book Discussion: Foundations of Software Testing: ISTQB Certification
• Proactive Testing Puts Agile Test-Driven (and Other) Development on Steroids
• Short-Cycle Development during Test: Case Study for a Legacy Enterprise IT System
• The International Software Testing Qualification Board Advanced Syllabus

Look over the full list of webinars and maybe purchase some. Let us know what you’ve learned, what you’d like to see us offer, and what we can improve.

Agile is an up and coming methodology, already in adoption many places and even used here at Computer Aid. The question becomes what role does QA play in Agile.

In his article, “What Agile QA Really Does: Testing Requirements,” Steve Berczuk argues that teams transitioning to Agile have a hard time knowing what to do with their QA teams because the focus is on developer testing of code.

Although QA teams in Agile should not become the sole testing source, they can assist in testing things that developers may not be completely able to test. In addition to catching coding errors that slipped through, a quick QA team may be able to find unexpected paths through the application and identify problems on these paths. Steve calls this exploratory testing.

In Steve’s words:

If you have a QA team, and they are doing exploratory testing, they are really testing:

Requirements: Finding interactions between features and components that were not defined or understood when coding and developer testing began. 

Developer Tests: Identifying where developer tests were not as good as they could have been (and how they could be better). This is a good use of automated “integration” tests.

Systems tests that might be hard to test in a developer context. This might be the one set of tests that are the primary domain of QA. Read More>>

So what are your thoughts? How can a QA team provide value on an Agile project? We look forward to talking about this with you on our blog.

First – I am a big fan of open source tools.  This is a great way to build real solutions in a collaborative way.  However (and that’s a big however), don’t put all your “eggs in one basket”.  Meaning, don’t count on open source tools to address all of your organizations needs for Test Management, Functional Automation and Performance and Scalability testing.

Let’s start by what they these tools are called -   “Open Source Testing Tools”.  This means that you as an organization are able to use them with little or no procurement involvement.  It is important to understand that you as an organization are obligated to not only be an active participant of ideas and collaboration but frankly the maintenance and support of the tool itself (particularly if you bring into your own organization).

Here are a few tools out there that you may what to know about.  Free to offer your opinion on these open source tools.

Functional Testing:

Canoo WebTest

• Development-focused solution for web applications producing HTML (JSP, PGP, etc)
• Scripts written in java or xml
• Provides detailed reporting

Selenium

• Flexible solution for web applications with strong support for JavaScript but limited reporting
• Record/playback or written scripts in “Selenese”
• Scalable for larger testing efforts

Performance Testing:

Apache JMeter

• Pure Java desktop application designed to load test functional behavior and measure performance
• Scripts written in Java
• Supports web, SOAP, LDAP, Database, Mail, etc

OpenSTA

• Windows-based application for performing and measuring scripted heavy load tests
• Record/playback or written scripts in SCL
• Supports HTTP/HTTPS; only manual correlation

There are lots of Software Development Lifecycles (SDLCs) out there. The selected SDLC affects everything from requirements gathering to QA and testing. So just how does the selected SDLC affect what you do for QA?

In the article by Eric Mumford titled “How is QA different for Waterfall, Agile, and XP?”, the author looks at each of the methodologies and expresses his opinion on how QA works. I have included excerpts of his opinions below. We encourage you to read the full article.

Waterfall: “The QA cycle involves authoring manual tests to match and cover the product requirements, authoring automated scripts to test the product features, updating regression scripts to test the defects that were found in past builds, and to run performance analysis on the build.  … It [sic]  requires larger toolsets to track test cases, testing results, performance testing, and automated scripting.  Waterfall … requires a heavy investment in tools to get the job done because of the copious amount of work that needs to be tracked.”

Agile: “QA focuses on writing automated scripts, usually using an open source test framework, against the software product as the features are being built.  QA prepares a performance test and adjusts the navigation scripts and virtual user balance ratio as necessary based on their estimation of the features in that iteration. When the software is delivered, QA often has only a few days to complete testing rather than weeks or months.”

XP (Extreme Programming): “Only the most technical QA Managers and QA staff members will survive in XP.  QA staff must have a working knowledge of shell script, Ruby/Python, and the deployment process, as well as SEO and ad-ops for web software.  QA will usually work entirely open-source in these environments, implementing tools in a “fast and loose” manner as needed.  Changing direction completely is common.  QA must write the minimal set of functions required to provide an automated test bed, code coverage metrics, manual cross-browser testing, and performance analysis and be able to do so in hours rather than days.”

What are your experiences? Have you been locked into a particular SDLC, and how has the QA used matched or deviated from Eric’s perceptions? Leave your thoughts and comments on our blog.

The growing popularity of utilizing the Agile Development methodology to develop systems has changed the way software is tested.  

However, you also have to ask yourself the question?  Are you ready from a QA perspective?

Let’s think about QA as it relates to Agile the questions you need to answer for yourself. If you say ‘no’ to any of these questions then you do really have to consider whether your ready.

• Do you have a ‘automatic’ regression test that can be run on a night basis?
• Is your build process automated and then tested against?
• Have you developed an “integrated” testing team that can lead and test successfully during a sprint?
• Have you looked at your current QA Tools and will they work in an Agile methodology?
• Are your ‘QA artifacts” right weighted for Agile?
• What important QA Metrics are important and operational for successful Agile sprints?
• Is your team makeup right for Agile?

These are just some of the things to consider.  If you said “NO” to only few of these questions then you probably will not be successful 

The growing popularity of utilizing the Agile Development methodology to develop systems has changed the way software is tested.  

However, you also have to ask yourself the question?  Are you ready from a QA perspective?

Let’s think about QA as it relates to Agile the questions you need to answer for yourself. If you say ‘no’ to any of these questions then you do really have to consider whether your ready.

• Do you have a ‘automatic’ regression test that can be run on a night basis?
• Is your build process automated and then tested against?
• Have you developed an “integrated” testing team that can lead and test successfully during a sprint?
• Have you looked at your current QA Tools and will they work in an Agile methodology?
• Are your ‘QA artifacts” right weighted for Agile?
• What important QA Metrics are important and operational for successful Agile sprints?
• Is your team makeup right for Agile?

These are just some of the things to consider.  If you said “NO” to only few of these questions then you probably will not be successful

Many organizations are trying to implement Test Automation as an approach to increase throughput and efficiency.  This is a “great idea” but with every “great idea” there has to be a strategic plan.   This strategy must include a scope, objectives, implementation plan, critical success factors and risk factors which are aligned to the organizations mission and goals.

These are the key areas that you as an organization need to focus on for implementing an automation framework

o Detailed Approach / Plan for developing and implementing an automation frame
o Required Processes and Artifacts for Automation
o Metrics to measure “ROI”