Test Case Design
Creating and maintaining an application means not only to design and code the application according to the requirements maintained in the requirements document, but also to design the test cases linked to these requirements and maintained in the test plan.
Test case design is about how to setup test cases so that the test cases are effective. Test cases needs to be designed well in order to ensure the quality of the application. Many different design techniquies exist to ensure that the functionality of the software and all features are tested well. The design has to ensure, that every aspect of the software is tested and that issues in the functionality of the application will be found.
With a proper test case design must be also ensured that the requirements are tested with a high coverage.
What is a Test Plan
A test plan describes the scope of testing an application or a system. It documents also the approach and resources used for testing. Test cases are documented and maintained in a test plan along with the test setup on what hardware and what operating system will be used and any data files necessary. A test plan should also include how the application we be tested and when automating, which test language will be used.
There may be different test plans for unit, integration and other test types. A test plan should include links to the project and configuration management plan.
What is a Test Case
A test case is one scenario in an application to be checked, whether it´s working like it should. It is a set of actions to verify the scenario described in the requirement. A test case is part of the test plan, where all test cases are documented.
Test Case Description
What do we need to describe and document a proper test case? As an absolute minimum a test case needs to have a description or summary, what the test case does, so what is the scenario to check. The "how" are the steps. A proper description what steps are needed to check the scenario, so everybody can run or automate the test case. Every test case also needs the data, which must be used to run the case and what is the expected result. One test case should have one result. You also need to have the requirement in mind, which should be covered with this test case.
Example of a standard test case:
ID | Scenario | Req ID | Steps | Data | Expected Result |
---|---|---|---|---|---|
1 | Check for valid login | 1 | 1. Goto login screen | User should login | |
2. Enter username | user: martin | Username entered | |||
3. Enter password | password: pass | Password entered | |||
4. Click on Login | User is logged in |
When a test case is executed against different environments, then these need to be documented in the test case execution alongside with the actual test result, date and time of execution and whether the test pass or fails.
This is a very simple test case. Most of the time test cases are more complicated with pre- or post-conditions, so everything must be documented. When the test cases are designed and run by different testers, the writer and executor should be specified as well. Pre-conditions do specify the status of the application, which must be achieved before a test case can be run.
Also testing a login does not mean, there is only one test case. Depending on the results there may be up to 6 test cases for the login.
An application is changing often during the lifetime, so it is a good practise to version not only the source code, but the test cases as well. The best way is to do this in the same Git repository and use the same release tags.
What is a Good Test Case
- A test case must be simple, transparent and understandable, have the KISS principle in mind when designing test cases,
- A test case must be exact and accurate,
- A test case must be small, isolated and atomic,
- Any test case repetition should be avoided,
- Make no assumptions on any scenario,
- Every action in the test should be a test step,
- Find an understandable language and use the same exact and consistent names inside the test steps (enter value in field, click on button etc),
- Don´t use any fill words (there is no need for "please")
- When using specific words, create a glossary, where a test executor can look up the meaning,
- Test only one thing at a time,
- Every test case must have always one result,
- When a test step does not change anything or does not have any effect, combine more steps to one, as every step should have an outcome,
- As much as possible coverage should be achieved, so testing techniques should be implented like boundary value analysis,
- The test designer should use his experise and use it for error guessing,
- A test case should not leave the test environment unusable,
- A test case must be repeatable and reusable,
- Positive and negative scenarios must be covered, but in different test cases,
- A test case can be used in an other test case, but be sure, that the pre-conditions do fit,
- Limit dependencied to other test cases,
- Don´t hardcode any data in a test case, use variables,
- A test case must be independant,
- A test case must be easy to maintain.
Albert Einstein: If you can’t explain it simply, you don’t understand it well enough.
What is a Bad Test Case
- The "God" case, which tries to test everything in one case,
- "Magic Test" is a test case weird steps going from A to C without B and makes things complicated,
- A test case with too much accidental complexity,
- Test case does not have a clear result,
- Inconsistent expressions through one or more test cases, no clear language,
- extensive calling of other test cases,
- A test step does not have an effect or any added value,
- Test case for dummies by explaing every step with full navigation,
- Test case having only a title and no steps,
- More than 12 steps in a test case are usually too much,
- Redundant Test case which are already created by other testers (lack of documentation?),
- An instable test case should be avoided at all, try different approaches.
Tips for Writing Good Test Cases
- Follow the topic "Good Test Cases",
- Avoid the statements in "Bad Test Cases",
- Things to consider for a good test case:
- Is the test trustworthy?
- How much will it cost to write and maintain the test case?
- Is the test reliable?
- Is the test targeted to determine the state of the application?
- Document your test cases very well in the test plan,
- Let others review and sign-off your test cases,
- Refactor your test cases often and invalidate old test case for old application versions,
- Follow the test case design techniques for
- Boundary Value Analysis,
- Equivalence Class Partitioning,
- Desision Table,
- State Transitions.