In Scrum, each sprint produces an increment, which is a partial and potentially releasable product. To be releasable, the increment must meet all acceptance criteria and pass different categories of tests. Consequently, each sprint should consider all testing activities required for a releasable product. Unfortunately, during my work on different projects, I have observed that many teams focus only on a subset of testing activities during the sprint.
In this article, I introduce the acronym PURIFF as a concept containing all test categories to be conducted during the sprint. The Scrum team can use the PURIFF acronym as a checklist to determine which categories of tests are relevant in the given context.
Each element of the PURIFF acronym represents a category of tests:
- P stands for performance testing.
- U covers unitary testing.
- R deals with non-regression tests.
- I represents integration testing.
- F is for functional tests.
- F covers non-functional tests.
PURIFF testing scope
The scope of PURIFF includes the following categories of tests:
- Performance tests. Performance tests ensure that the system response time is acceptable for the users. The tests consist of measuring the applicationís reaction time to the different requests. There are different tools (commercial and open source) that can be used for performance tests: HP LoadRunner, IBM Rational Performance Tester, and Apache JMeter are examples of performance testing tools.
- Unitary tests. Unitary tests are written by developers to test individual classes. They fall into the test-driven development, or TDD, strategy. Different solutions exist to write and run unitary tests automatically. JUnit and NUnit are among the most popular tools for unit testing.
- Non-regression tests. The objective of non-regression tests is to ensure that new defects haven't been introduced after the modification of the code. Indeed, each code change is likely to introduce regression on the existing system, so retesting the already-tested system becomes necessary. Regression tests are a tedious activity, especially in an Agile context, which is characterized by continuous changes and frequent deliveries. They are also time-consuming when done manually. I strongly recommend automating them. Even if automation has a cost, the return on investment for the project is high. FitNesse, Selenium, Microsoft Test Manager, and HP Application Lifecycle Management (ALM) are examples of tools that the team can use to automate non-regression tests.
- Integration tests. Integration tests allow us to verify that the modules (a set of classes) work well when put together. They are usually run automatically on the continuous integration environment.
- Functional tests. The scope of functional tests is to check that the system behaves according to what has been specified through the elements of the product backlog (the acceptance criteria). The tests consist of testing each new feature of the increment against the corresponding acceptance tests. These tests can be run manually or automatically.
- Non-functional tests. Non-functional tests cover the non-functional aspects of the application. These aspects are often defined as non-functional requirements (technical user stories). For example: scalability, robustness, security, portability, stress testing, and so on.
In working with teams, I've found that using the acronym PURIFF is highly useful. It functions as a concept that contains all the testing activities that have to be considered in a sprint (iteration). Even if all the increments are not intended to be deployed on production, the Scrum team can use PURIFF as a checklist to determine which categories of tests are relevant to the context at hand.