Here’s a *hypothetical* issue.
There is a performance issue in the current release. Its difficult to find the cause of this issue because the performance tests haven’t been run in roughly 10+ releases (with anywhere from 10 – 100 changes per release). My idea is to automate the process of running the performance tests in each release. However, this automation took a bit longer than expected to get running.
My (again *hypothetical*) response to the automation taking too long would be to simply perform the tests manually, and then run through the results with a set of scripts that I’d already written.
In the end, this automation would be SOO beneficial to have, but it could take longer than is normal for a task to be completed.
Is that the right response to what could feel like a rapidly diminishing ROI on a proposed solution?
I wonder about the relative terms you use in your question. In particular, you said “rapidly diminishing ROI”, which I’m not seeing within your question. Difficult to find bugs can greatly tarnish a company’s reputation. So I wonder if you’re looking at the full scope of this benefit equation.
You have two principle drivers here, Cost and Benefit.
Cost is amount of development time to automate the performance tests.
Benefit is the amount of improvement in client satisfaction from avoiding the next performance issue.
All you need to do is weigh those out for a few more hypotheticals.
How bad was the reaction from this last release?
Do you know for certain the performance tests would have caught the issue?
Lay out best, realistic, and worst case estimates for coding the automation.
If your worst case cost is justified by the benefit, then go with it. If it’s marginal, discuss it with your team and determine where the inflection point is at.
I could now write an essay about how to do cost/benefit analysis of automation projects, but this image says more than I could ever write about this subject:
source: http://xkcd.com/1205/
2