Testing Integrated in Development – Collaboration and Frequent Communication
The traditional waterfall software development model is a sequential process with distinct project phases;
specification - design - implementation - test - maintenance.
Testing is explicitly visible as a separate phase, but late in the project and with no or little influence on the preceding phases.
As opposed to the waterfall model, in agile development, testing is an "invisible", integral part of the entire lifecycle, where testers, developers and business people all have a role in testing. Frequent communication is of key importance and helps removing defects early, thus providing higher quality products with lesser effort. Test documentation is kept lightweight (it's produced only if it is "of value"), and testing is more exploratory.
In Scrum, the most popular agile methodology, requirements are expressed as user stories. The user stories should be produced by business people, developers and testers in collaboration. Including testers already when producing user stories can greatly improve the quality of the stories because testers normally are good at identifying missing details, seeking clarification of what is required, ensuring it is testable and confirming acceptance criteria.
Test Automation
Two agile keywords are incremental development and responding to changes. Incremental development means that new functionality is added on top of the already existing functionality at frequent intervals, typically as 2-4 weeks sprints in Scrum. Responding to changes means that changes are expected, welcomed and managed at any time, though early changes of course are less disruptive than late changes. In total, this means that testing is a more or less continuous process in agile development.
Continuous testing require high level of automated tests, especially at lower test levels. Regression testing quickly becomes boring to testers, and the more of these tests that can be automated, the better. As a minimum, a continuous integration process should be automated and performed at least once per day. A continuous integration process typically comprise the following steps:
compile the code - build the system - deploy into test environment - run automated tests.
Test Levels and Test Types
The main test levels are:
- Unit testing
- Integration testing
- System testing
- Acceptance testing.
The level of automation will typically be high at unit test level and low or non-existing at acceptance test level.
At each test level, different types of tests are relevant. The agile testing quadrants below align test levels with the appropriate test types and helps to ensure that all relevant test types and test levels are included in the lifecycle. Further, it shows which test types should be automated and which should remain manual, whether the tests are business- or technology-facing and whether they support the team or critique the product.
Exploratory Testing – Use of Test Charter
Agile development emphasizes exploratory testing, but this does not mean that testing is randomly performed without any plan. Exploratory testing is an approach, where many aspects are taken into account, such as techniques to use, time available, risk and testers' skill, and the key ideas are to question the product from a testing perspective, to create an overall mission and to build a mental model of the product.
Exploratory testing follows a Test Charter which is a one-page plan for a testing session describing the purpose of the test, what should be tested, how to test (use of tools and techniques) and the expected duration of the test. During the execution of a charter, it's important to keep to the alleged time, remain focussed, take notes and follow-up afterwards. The effectiveness of exploratory testing is heavily dependent on the tester.
Pitfalls
For agile development to become a success, some significant changes may be required. The pitfalls are many, e.g. the business people are not involved, testers are left out too long, old procedures are still used, the same amount and type of documentation is produced, the developers do not perform any unit integration testing or too much focus is kept on automation
Agile development require changes in attitude and culture, otherwise already existing dysfunctions become even more visible. Agile is both a mindset and a process and failing to implement the required changes only lead to faster chaos and faster delivery of bad products.
