Recently, the London 2012 website was forced to accept sales until 0100 BST on Wednesday 27th April, pushing the time limit forward from the original 2359 BST Tuesday 26th April deadline. An hour before the midnight deadline, some users were met by a holding message on the website which read: “We’re experiencing high demand. You will be automatically directed to the page requested as soon as it becomes available. Thank you for your patience.”
This one instance highlights the need for performance testing of an application, but just what is performance testing?
Performance testing itself is an umbrella term for the testing that is performed, to determine how fast various aspects of an application perform under a particular workload. It can also serve to validate and verify other quality attributes of the system, such as scalability, reliability and resource usage.
However, the term “performance testing” is often used to mean one or more of the different types of performance testing.
Performance testing includes the following;
Load testing is the simplest form of performance testing. A load test is usually conducted to understand the behaviour of the application under a specific expected load. This load can be the expected concurrent number of users on the application performing a specific number of transactions within the set duration. This test will give out the response times of all the important business critical transactions. If the database, application server, etc. are also monitored, then this simple test can itself point towards any bottlenecks in the application software.
Stress testing is normally used to understand the upper limits of capacity within the application landscape. This kind of test is done to determine the application’s robustness in terms of extreme load and helps application administrators to determine if the application will perform sufficiently if the current load goes well above the expected maximum.
Endurance testing is usually done to determine if the application can sustain the continuous expected load. During endurance tests, memory utilization is monitored to detect potential leaks. Also important, but often overlooked is performance degradation. That is, to ensure that the throughput and/or response times after some long period of sustained activity are as good as or better than at the beginning of the test.
Spike testing, as the name suggests is done by spiking the number of users and understanding the behaviour of the application; whether performance will suffer, the application will fail, or it will be able to handle dramatic changes in load.
Configuration testing is another variation on traditional performance testing. Rather than testing for performance from the perspective of load you are testing the effects of configuration changes in the application landscape on application performance and behaviour. A common example would be experimenting with different methods of load-balancing.
Isolation testing is a term used to describe repeating a test execution that resulted in an application problem. Often used to isolate and confirm the fault domain.
So next time a customer mentions the need for performance testing, ask them what it is they want to do? As it may well be the case they want to do more than one type of performance testing!