Return on investment is, according to Investopedia, a popular metric because of its versatility and simplicity. Essentially, return on investment can be used as a rudimentary gauge of assets profitability. In addition, ROI can be straightforward to calculate and interpret. The simple formula for ROI = (gain from investment - cost of investment) / investment costs. In the formula, gain from investment refers to the realized savings.
ROI Performance
Businesses today live and die by user experience. Yet, because of the increasing complexity of their services and the pressure to deploy quickly, companies don't take the time to ensure that their applications deliver acceptable response times under actual and future growth load patterns.
However, some IT leaders learned from previous mistakes and integrated performance engineering successfully in their DevOps chain. They start early during the design and development stages with performance analysis, automate those tasks, review the results and eliminate hotspots in the construction phase. This post will demonstrate how you can calculate the return on performance investments.
Costs of performance investments
Responsive applications, which are robust against minor changes and deliver acceptable response times under spike load situations, are not free. Things would be so easy if there would be a performance feature, which you could buy on the next corner, integrate it, and suddenly all hotspots have been eliminated. Unfortunately, there is no permanent cure for performance. Organizations hire engineers, buy tools and set up performance testing platforms on their infrastructure.
Professional businesses rely on commercial load testing suites. They pay an initial license fee and yearly maintenance charges to get the required support and product updates. In addition, load injection requires powerful and exclusive load agent machines, which leads to additional annual costs. In recent years SaaS-based load testing platforms appeared on the market. Businesses realized that those result in less load testing efforts because there are no initial fees, and they charge for the actual usage.
But, there is one more often ignored cost driver in the performance testing space. Your engineers will identify long-running requests, applications that crash, memory leaks, or other defects. The analysis, implementation of fixes, and deployment of the improved code will cause additional efforts. Research has shown that late detected issues are much more expensive. For example, a defect in production leads to a factor 1000 higher efforts than on development stages.
Savings of performance investments
Reputational gains, trust in IT, more time for other tasks due to minor troubleshooting benefit load and performance tests, but it's almost impossible to calculate their financial impact. Personally speaking, I appreciate fast-loading web pages, and I assume that those providers manage their IT services adequately, but the cost impact would be tough to calculate. The same principle is valid for reputational wins or time, which could be spent on other tasks. Therefore, I will not consider these factors when I calculate the return on performance investments.
On the plus side are increases in revenue and earnings due to earlier detected performance defects. Amazon, for instance, has demonstrated that a 100 ms speed improvement pushed their sales by 1 %. When users experience slow loading web pages, they often abandon and spend their money on a similar website from one of the competitors. Load and performance tests help companies to fix such slowdowns early in the lifecycle. Those fixes on the pre-production stages are easier to implement and result in additional savings.
Calculation of ROI Performance
Have you ever calculated the actual costs and savings of your load and performance testing services? I was curious how this would look for an on-premise load testing solution and made some efforts to calculate and compare both.
In my sample, I've used the following key figures:
1000 concurrent user load testing tool license
120 test executions per year
800 HTTP based and 200 real browser-based user in each test run
8 load injection machines
3000 $ annual costs for each load injection server
120 performance defects per year
20 % fewer defects on production due to load testing
2 performance engineers earning 60.000 $ annually
Not considered in this calculation are
Increases in sales due to less abandon rates
Efforts to fix identified performance defects
ROI of on-premise based load and performance testing platforms
Companies deploy such load testing suites on dedicated servers hosted in their data center. Some open-source solutions have limited user simulation features on the market, but professional businesses rely on commercial platforms. The vendor of this load testing suite charges an initial license fee of 100.000$ and an annual maintenance fee of 20.000$.
Two engineers are responsible for load and performance testing in this company, and they identify 120 performance defects per year. The company uses 8 dedicated load injection machines, resulting in additional expenses of 3000$ annually per machine. On the savings side, there are 20 % fewer defects in production.
The chart below shows how expenditures, earnings and the corresponding ROI progresses over a period of 5 years.
My calculation shows that an organization that relies on an on-premise load and performance-testing platform realizes a 33 % return on their load and performance testing investments after five years.
Conclusion
Load and Performance Testing is an investment. The break even is reached in the second year and after five years an excellent 33 % return on Performance Testing is achieved. There are very little investments that outperform ROI Performance.
Be open minded and don't waste money by neglecting this opportunity. We are here to help. Contact us if you are interested in a tailored calculation or our performance engineering services.
Keep doing the great work! Happy Performance Engineering!
Comentarios