In the previous post, I wrote about why performance is so important and on how to create strong non-functional requirements. Let’s focus a bit more now on the implementation of load and performance tests.
This step in the development lifecycle is where the real work for performance-engineering teams begins. You’ve already agreed on the overall process; your non-functional requirements are aligned, and your developer has started on the coding. The performance engineers are now fully occupied with load simulation, performance analysis and tuning support activities.
The term load and performance testing is widely known. It captures the most important duties of the technical-testing teams and is intended to validate performance requirements at the preproduction stages. The major steps in load and performance testing are:
create a load-testing concept
set up performance monitoring
run API-based load tests
set up automated performance validation
run end-to-end load tests
conduct the results analysis and provide reports
raise any defects
run the tuning support and execute re-tests
Load and performance testing involves a lot more than a simple performance measurement, in fact. It means that engineers simulate production, user, and request volumes on your new application as well as checking how well the entire system can handle the average and peak-production volumes. During the usual performance test execution, the engineers also conduct testing on response times, error rates, concurrency problems, hardware sizing and log-file handling in addition to various other operational readiness checks. Perhaps we should rename the load and performance test activities to reliability testing—as this describes the performance engineer’s spectrum of activities more accurately. The term also conveys the importance of this testing to the management level more effectively. After all, reliability is key in every business application. Performance often tends to be done as an afterthought.
Further, load and performance testing—like its cousin security testing—tends to be neglected. Both disciplines deal with technical requirements and require very specific technical, rather than manual, skills. The specialized skills involved means that organizations are often overlook both types of testing. For those of you who appreciate the huge benefits that load and performance testing can bring to your business, I’ll now give you an overview of how to prepare and execute this testing.
Load and performance testing concept
Personally, I never execute a load test without first having an approved summary of the intended test. For this, all parties involved —such as project teams, architects, and the application owner—must agree on this performance-testing approach, as formalized in the conceptual document. You don’t need to write much. Just focus on your requirements and clearly specify how you intend to validate them. The major elements of this conceptual document include:
performance requirements such as user volumes and response time expectations
load patterns to be simulated
system chart of the application under testing
description of the test environment
test-execution planning
test reporting and key performance metrics
defect-handling details
team and responsibilities
risks
Make sure your load and performance testing concept will be approved by your project lead before you proceed with the test execution.
Set up performance monitoring
Engineers tend to focus on load simulation and tuning, and delay the most important step—performance monitoring. When done properly, this gives you fantastic insights and will help you detect the root cause of your performance hotspots. Don’t just settle for a log-file or infrastructure-based monitoring. In your role as a performance engineer, you’ll need much more detail than log files or system-resource utilization can give you.
Performance monitoring is essential for two reasons:
You can’t trust log files because you have no control over their content
You need to investigate application-specific topics such as transaction flow, concurrency issues, slow SQLs and slow method calls
Start with the setup and integration of performance monitoring early in the development lifecycle and give access to all teams from Dev to Ops. Your teams will soon realize how the valuable insights they get will show them what’s going on in their applications. Choose a powerful application and user-experience monitoring solution that collects all transactions.
API-based load testing
Your testing approach has been approved and your applications have been equipped with a performance monitoring solution. Now you can move on to the implementation and execution of API-based load tests. As already mentioned, load testing should be integrated early in the development life cycle. The frequent changes in applications at those stages have a massive impact on the automated testing activities. If you choose the wrong testing approach, your attempts to adjust the load-testing scripts to the changed application will be endless.
Fewer changes are made in the interfaces and services than in the user interface. I therefore, highly recommend choosing an API-based user-simulation approach for your performance validation in the early development stages. Implement your selected API-based testing scripts, parametrize your input data and configure the required test configuration according to the test execution plan in your test concept.
Set up automated performance validation
Each change in your application, configuration, or infrastructure can have a huge impact on your application’s performance. The later your engineers identify performance slowdowns, the more effort will be needed for remediation activities. Early performance validation helps you to address such problem spots quickly. Obviously, you can’t do such daily or hourly load-test executions automatically.
Automate your API-based load tests with any scheduler that’s available in your firm. Make sure that the execution can easily be triggered and that key performance figures are readily accessible to your teams. Use a performance quality-gate configuration to validate the performance metrics automatically. This continuous performance baseline will help your development teams to identify breaks quickly and will also give them more time to fix problem spots.
Implement end-to-end load tests
Research shows that a large number of slowdowns can be attributed to the client layer. Conducting load tests for services and interfaces works well for early performance validation, but won’t give you any insights into how the new system performs end-to-end. Problem spots often appear after real use cases are executed. The communication pattern and the data flow between the new services also need testing before you move an application to production. End-to-end load and performance tests simulate complete use cases for features such as a checkout or product search. Those measurements also include the client layer, which is often the cause of many performance issues.
Happy Performance Engineering!
Comments