There are many paths which could be followed with performance testing, and this post will attempt to break these down to provide a number of points that should be considered when setting up performance testing for your application. In particular we will focus on web applications, but similar principles could be applied to other applications.
What type of performance testing are you doing?
In general performance testing is a measure of responsiveness. However, it can also be a measure of responsiveness over a certain period of time. In a sense then there are two types of responsiveness that performance testing is interested in:
1. How long: That is how long it takes for a particular page to load, or a request to be sent, or a function to return. Or an overall Use Case to be completed as a workflow through a website. Or a client call to a backend to return.
2. Memory profiling: In a sense this could be ‘how long’ after ‘how long’? I.e. We know doing an action once or twice is responsive from the type of testing we do with ‘how long’, but we also need to know if this stays the same over time. Profiling memory over time can help to indicate if there is a problem now, and/or one building for the future.
N.B. You may also be thinking that adding more users should also be considered when measuring performance. Yes it should. But strictly speaking this would be Load Testing rather than Performance testing. It is likely that a lot of the same techniques and tests could be used. The line between the two is not always going to be clear.
We may know that doing something once is responsive, but what if we do it 100 times, or 1000. However, this also requires some user profiling, or use case analysis. If the function will only be used once or twice a day there is little value in proving that it is still responsive after 1,000 repeats. However, if the function will run many times a minute or even second then it needs to be seen to respond within this time, and be seen to remain responsive over time.
What level of Performance are you interested in?
- Application level
- Component level
- Unit level
In general performance testing often is not done until you get to the application level. At which point complex causes and fixes can be suggested if the application is found not to be responsive. Instead like development in general it is better to test units first, then the components made up of those units, and then the application that is built from those components.
Which presents another issue. How to know from the overall application level performance requirements what each unit’s performance requirement should be?
Nonetheless having unit level focused performance tests will make it easier to ‘zoom in’ on an unresponsive area of the application during later application level testing if necessary. Assuming that this wasn’t done then you need to make the application level testing focus on particular components at least at one time. Thus providing another way of ‘zooming in’ on a specific area of the application. This could be done by writing separate tests, or by ensuring that it is clear in log files where one function ends and another begins. This leads us on to how you will do the testing.
In part 2 we will consider tools, environments and Observer effects for performance testing.