[danso]

This is true in finance and in other data-heavy fields as well. I've been shocked at the kinds of Excel sheets that, with a mess of spaghetti VB code written by someone long gone, factors into trades worth millions...sure, it "works"...but besides the very minor question of code elegance, who knows what optimization of returns could be made if the code wasn't such a fright that a knowledgable partner could tweak and experiment with it? Or that it was abstracted enough to be applied to the other kinds of trades that the firm is making (but hell what do I know, I'm not as rich as my hedge fund friends)?

What's particularly annoying is working with analysts who have a system of pasting SQL scripts from a (hand-labeled-versioned) text file to perform the necessary data-munging/pivoting for in-house use...their SQL work is, to be fair, so much of a leap forward from however such bulk data work was being done previously that they take offense when I offer to help them automate the work...as if their system of hand-pasting/executing scripts, then eyeballing the results for an hour to spot-check it, was inherently more reliable than a batch script with well-defined automated test parameters...What they fail to see is that it's not just about faster/better error-checking, but it's about more flexible analysis and output. Once the process has been abstracted, instead of producing one "clean" giant database that is faceted along one dimension (time, perhaps), the script can loop through and spit out a variety of useful permutaitons, which would be impossible/insanity if you stick with the hand-tweaked process.

That's the problem I see with the OP...A scientist can recognize when something seems to work, when it comes to the domain of programming and structure, but "what works" may simply be "what seems to work better than what I did last time"...which is not a foolproof standard of evaluation