How to determine test coverage
A while ago I answered the following question on SearchSoftwareQuality.com’s Ask The Software Quality Expert: Questions & Answers.
The project that I’m working on is to create regression test scripts for applications which are migrating from another location. This test team has mainly functional (system test) scripts and there are many applications that we are taking over. How would you approach the regression scripting considering we have many applications and limited resources?
Here is a clip from my answer:
I follow the same basic steps when I think about regression testing as I do for any other type of testing. I try to force myself to think about coverage, risk and cost. I’m always looking to evaluate what tests can best address my most important risks with the most coverage given the time, tools and resources I have.
Understanding what you have to cover
I would encourage you to start off by making an outline of all the things you could potentially cover in your regression testing. [...]Understanding your risks
After you have a list of what you might want to cover in your regression testing, start thinking about the specific risks that you’re concerned about. [...]Putting your risks and coverage together (chartering your work)
Once you know what you want to test (coverage) and why you want to test it (risk) you’re ready to charter your work. Chartering is the activity where you put it all together in meaningful terms. [...]Prioritize the work and figure out what you can do
Once you’ve chartered your work, you should have some idea of how much work might be in front of you if you wanted to test it all (or everything you could think of at least). [...]At the end of all this, you’ll have a list of charters for each application that you should be able to execute given the resources you have. Following some sort of schedule that makes sense for your team, get in the habit of incrementally reviewing the coverage, risk, charters and priority you have for each application. You’ll find that what you want to test and why you want to test it will change over time.
You can find the full posting here.
The difficulty I have with coveage is understanding what you have to cover, especially given there are an infinite number of tests.
If you want to test a link within a website which goes to another page you could test the following (without thinking too hard):
1. Usability - is the link clearly visible to the user?
2. Functionality - does it take you to the correct page?
3. Performance - does it take you to the correct page in a *reasonable* time?
4. Security - does the link open a security hole within the application?
When you have numerous tests for even the smallest of functions within an application how can you possibly articulate what kind of coveage you’re getting to a project team?
I do it a couple of ways:
- I talk about what areas of the application I’m covering: reporting, core platform, client scripting, voice, etc…. That provides an indication of /where/ I’m testing.
- I talk about what types of risks I’m covering: usability, performance, capability, security, etc…. That provides an indication of /why/ I’m testing.
- I talk about what specific features I’m testing: stories, tickets, etc…. That provides an indication of /what/ I’m testing.
Those three dynamics are important because they tell the story of my coverage. “This is what I’m testing. This is where it lives and what’s around it. This is why I’m testing it.”
I have a website which is having number of links in each page.so in this case…
1) I know that i have to check each and every link whether it is taking us to correct [age or not and so on.
2)The same link is displying multiple times in one page which all are opens the same page.Do we need to tets eaxh and every link ?
3) If click on that link,then it will open anothre page and that contains multiple links….
so, whoch approach we need follow …to say,that i covered all the things.