I’m developing a numeric application (nonlinear optimizer), with a zillion knobs to tweak and rising.
It’s not my first foray into this domain, but this time there are even more variables in the code and I’m on a tight schedule. Don’t want to waste time fiddling.
Days or even months can potentially be wasted adjusting variables, recompiling, and reprocessing benchmark datasets. The resulting data is viewed and trouble spots are checked. The overall quality of the solution is reported by the program but the meaning of the report could change over time. (Numeric units for the report are one thing I’m trying to nail down.)
One main problem is organizing result files to identify each with specific code changes. Note taking can be a pain, is there software to help with this?
Are there agreed best practices to making this kind of development cycle reliably move forward? The solver package converges to its optimal solution with mechanical determination, but I’m all too familiar with the way an excess of design decisions can mire development.
oh boy there is a lot going on in this question, and I’m not sure I understand all of what you are asking but lets try:
Days or even months can potentially be wasted adjusting variables, recompiling, and reprocessing benchmark datasets
at the very least I’d suggest pulling the variables out into a config file so you do not need to recompile to change them. Ideally you would also write some benchmarks and maybe use a technique such as hill climbing or GA to programatically refine the variables towards an optimal solution
(Numeric units for the report are one thing I’m trying to nail down.)
Do you mean units of measure? there are libraries available to help with these in most languages. F# even has them build into the language (though not for reporting)
One main problem is organizing result files to identify each with specific code changes. Note taking can be a pain, is there software to help with this?
the obvious solution here is to have your benchmark results committed to source control with the changes that made them, you could then even use commit hooks to signal any worsening in benchmarks
3