Test Automation – Expense or Investment?
A transition from manual testing to automated testing is a project on its own. Hiring resources with new skillsets, introducing or developing new tools, requests for additional infrastructure and environments, is a guarantee. All this translates to a bigger upfront investment in comparison to manual testing. Without fixed expenses nor clear predictable gains at the start of a transition, calculating the ROI of such endeavor is not a simple task. Here we will point to several less obvious aspects which need to be taken into consideration when estimating the ROI. What to balance all the costs with?
True ROI
The most basic and naive calculation of ROI is to estimate the time automation saves on any repetitive task. For example, if it takes 1000 hours for a team of 10 testers to regress your existing product, or 1000 hours for a team of 5 automation engineers to write automated tests, you would save money during the second regression run already. This also takes into consideration that automation engineers cost more that manual testers.
This basic calculation is naïve, because it doesn’t account for test maintenance. The initial number of tests will increase with new functionality and existing tests need maintenance efforts over time. As tests are vulnerable to code changes one needs to plan time to keep them up-to-date. It is vital to address these issues by choosing tools that minimize the time to write and maintain your tests so your team doesn’t spend the majority of their time updating tests rather than extending the coverage.
Production bugs are more expensive
Because test automation brings savings at every execution, in order to maximize the ROI it needs to execute as often as every build. Scheduling test automation as part of Continuous Integration, developers are immediately notified about failed tests letting them fix it sooner rather than waiting for it to be reported, triaged and then resolved as has been done in the past. The longer it takes to identify a bug the more expensive and harder the fix becomes. The average cost of fixing defects found in production is 5 times higher than those found while still unit testing. This way automated testing addresses the largest unaccounted cost in software development, fixing defects.
Test data anyone?
Another less obvious benefit is test data preparation. It is estimated that Automated Testing can reduce the time spent on test data preparation by 80%. Automation can preload environments with required data such as accounts, policies or claims in a specific setup. This allows your manual testers to then focus on testing instead of populating data. Going back to ROI, the more it takes to execute a manual test the bigger savings you will have.
Justifying test automation on costs only is unfair. Automated testing reduces the project’s failure risk which ultimately reduces the company’s failure risk. By sticking to low quality you risk losing your current customers, missing market opportunities because overwhelmed manual testing hold up development, and undermining trust in the market putting a stop to future growth.
And there’s more!
When making a case for test automation do not forget about the following extra benefits:
- Faster delivery
- Extending your testing coverage
- More releases per year, quicker time to market
- Increasing the capacity of your test team
- Avoiding the problems of an ever growing team of manual testers
- Test automation ROI is very hard to calculate. It cannot be evaluated in a short time frame by looking at costs only. The qualitative and quantitative benefits of automation are a long term investment and crucial for a company’s success.
Automated testing results in a high quality, highly adaptable product that far outweighs the perceived cost of automated testing.
Jarek Paluch
Senior Software Architect
Watch or read our other posts at Kimputing Blogs. You’ll find everything from Automated testing to CenterTest, Guidewire knowledge to general interest. We’re trying to help share our knowledge from decades of experience.
