How fast is fast enough, why does it matter, and by what standards?
Questions like these dominate requirement-gathering conversations, and there are many reasons for them to end there. Usually, it all starts because of unrealistic expectations or no performance requirements in the first place.
Six response time categories
Of course, every user wants an instant response. Ian Molyneaox pointed out that we could divide response time expectations into six categories.
< 15 seconds for conversational interactions
< 4 seconds if the information must be retained in short-term memory
2 to 4 seconds for operations that require a high level of concentration
< 2 seconds when a user has to remember information through several responses
< 1 second to allow easy navigation, such as writing text on the computer
< 0.1 seconds to enable instant processing, such as pressing a key
Performance and Engagement
Other research has shown that engagement suffers from customers and impacts sales.
Even a 1% delay in page load can cause a 7% drop in conversions
40% of people abandon a website that takes more than 3 seconds to load
35% of customers will be less likely to purchase your products or services if your website performance is poor
Usability Engineering
There are also quite helpful response time guidelines in the book Usability Engineering, Jacob Nielsen, ISBN-13: 978-0125184069
0.1 seconds gives the feeling of instantaneous response — that is, the outcome feels like the user, not the computer caused it. This level of responsiveness is essential to support the feeling of direct manipulation (direct manipulation is one of the vital GUI techniques to increase user engagement and control).
1 second keeps the user's flow of thought seamless. Users can sense a delay, and thus know the computer is generating the outcome, but they still feel in control of the overall experience and that they're moving freely rather than waiting on the computer. This degree of responsiveness is needed for good navigation.
10 seconds keeps the user's attention. From 1–10 seconds, users feel at the computer's mercy and wish it was faster, but they can handle it. After 10 seconds, they start thinking about other things, making it harder to get their brains back on track once the computer finally does respond.
Are Performance Standards for everyone?
No: Do you provide monopoly or commodity services? The stronger your competition, the more likely clients are to move to another player after suffering performance issues. Another factor is switching costs. The lower they are, the higher the chance that performance problems drive clients to your competition.
Yes: Every business runs on software hosted in the cloud or local data centers. By ignoring all performance standards, IT infrastructure costs could spiral out of control and impact revenue.
How do you create your Performance Standards?
I recommend practicing end-game thinking. Create a mapping of your End-to-end transactions. Understand how your users will use the system, and specify the performance thresholds top-down from the end user perspective down to the web, application, and data layer. It makes a big difference if you have ten synchronous or ten asynchronous API calls in your end-user-facing transaction. Applying the same performance benchmarks/thresholds for your APIs is dangerous.
Alex Podelko published a detailed overview of how to specify performance requirements. His paper is available on this link
Keep up the great work! Happy Performance Engineering!