The question of how to build a testing environment is dependent upon the test plan and within this plan the test strategy. This defines not only the testing approach and testing environment but also sets the role of software testers by defining what testing to do. The scope of the project testing strategy is defined by the risk appetite of management who have to balance the risks of either too little testing with the potential risks of the software not meeting the clients requirements, and conversely too much testing and not meeting shareholder expectations due to higher costs.
The questions of what kind of testing to do and how much testing is enough are influenced by numerous factors such as project size and complexity, these factors collectively determine not only the extent of testing and the testing time window, but also the testing environment. In part, the risk appetite of management determines when to release the product and how error prone the software is likely to be.
A test strategy can be defined by selecting and ranking test factors, identifying the development phases and the business risks associated with the project and applying those risks to the risk matrix.
Testing strategy: Role of the Testers and the Scope of the Testing
The first step to building a testing environment is to determine the role of the testers and the scope of the testing necessary but it also requires management to make decisions on the following areas:
- A clear testing role
- Testing Policy (based on testing plan, defines what will be included in testing processes and communicates the role of testing to interested parties)
- Software Testing Support
- Methodology, Processes and tools for testing
- How many and location of testing labs
- Resource allocation
Testing strategy: Risk Management
The risks associated with managements decisions in regards to the above points can result in a product that is not up to specification. Risks include:
- Insufficient time and budget
- Inadequate test processes
- Software design failure
- Not meeting clients requirements
- Data issues
The whole point of a planning period for testing plan is to design a testing period the purpose of which is to mitigate the business risks that have been identified as well as to ensure software usability, -that the software meets the customers expectations (be tested for against application requirements). During this planning testing period all assumptions must be challenged to identify business risk factors.
Testing strategy: Customers Requirements
Since each testing team will have a slightly different purpose, they will have differing testing strategies. In general, terms, the strategy tests to see if the software meets the customers requirements, such as:-
- Does the software meets industry standards (compliance)
- Check software reliability
- Test to ensure Ease of use
- Service levels Requirements are met
- Maintenance is straight forward
- Integration points are as required
- Performance meets expectations
- Complexity of deployment is manageable
- Access Control meets the customers needs
Throughout every phase of the SDLC testing verification activities are carried out, which vary phase to phase. For each phase of testing, software documentation must be analyzed for testability and sufficiency. Test sets should be based upon this software documentation and it must be checked to see if it is consistant with previously generated software documentation, after which either refine or build new test sets fit for purpose.
Throughout every phase of the application lifecycle verification testing is carried out for which every action must be recorded and stored for many years. There must be complete transparency, when dealing with scaled Agile, in a multi team large project with millions of artifacts in, a tool must be utilized to track everything, a tool such Intland Software’s codeBeamer ALM.