[conductr]

> Programmable commands instead of a data grid would be huge improvement to quality...

Would it? At the end of the day, someone else still has to proofread and QA the commands/formulas/program or it's just blind trust that the decision is being made on. Trust (or ignorance) that the creator knew what they were doing and developed it in an accurate way before action is taken on the decision being made. The interface really makes no difference, it's the human component and "process" for creation that needs to be fine tuned. Things like the London whale situation was a process failure where one person had too much power to execute trades without oversight, review, QA, testing, etc. [0] All things that are pretty standard in a software developer's day-to-day but the rest of the world has not realized or adjusted to the fact that they are now software developers too.

[0] Excel wasn't the problem with the London whale at all, they made a mathematical error "modelers divided by a sum instead of an average"

[TAForObvReasons]

The tools are scapegoats.

The Reinhart-Rogoff issue was technically an error in Excel, but also an error by the authors for not actually verifying the results before publishing. It didn't hurt that their particular biases were in line with the results.

The technical problem can be addressed with more warnings and safeguards, but they are meaningless if no one uses them.

[conductr]

I hadn't previously read up on the RR issue. But after some surface level research, I would not say it was Excel as an issue. It sounds like the tool did exactly what they programmed it to. It seems like human error or choices they made to arrive at the conclusion they wanted; which seems to be speculated (or true, I only scratched the surface).

> While using RR’s working spreadsheet, we identified coding errors, selective exclusion of available data, and unconventional weighting of summary statistics. [0]

I'm not a fan of tools giving warnings for these types of "coding errors". Although a warning I can think would be nice is where math just doesn't work as expected. The recent floating point discussion [1] seems appropriate as it's just not very intuitive and as a programmer you need a pretty deep level of understanding to know that the resulting math is likely not accurate. But, it also seems to effect nearly every programming language and is not a quirk of one specific thing.

I'd be interested to read more if you have info outline the actual error within Excel. If there is some 2+2=5 situation, I'd be interested to learn about that. I feel like every time someone says "Excel error", it's actually "human error". It would be like if every car accident was a vehicle malfunction but we all know it's most likely an operator issue.

[0] http://peri.umass.edu/fileadmin/pdf/working_papers/working_p...

[1] https://news.ycombinator.com/item?id=30856434

[Psyladine]

Go farther. Would the results have ever been validated if the source of truth was not a universal format easily interpretable by millions?

[conductr]

I think what you're touching on is complexity and how many people tend to trust complexity because it's too difficult to validate and you must know what you're doing if you built something so complex.

Even in a corporate environment, I use spreadsheets to support big decisions every day, the format is as you say universal and easily interpretable, but many people never go that far. They trust that I did it correctly and take the "output" as truth. If I screw up big ($), it might cost me my job, but is it really my fault if nobody else even bothered to check my work?

[Psyladine]

Bikeshed effect writ large, I'm afraid.

It's not our fault, our brains are wired to find quick & dirty; simple not small is beautiful. But people also contemptuous of what they understand, and accept at face value what they don't. Michael Crichton, much disparaged in his later years for contentious political stances, had much of value to say in his Gell-Mann Amnesia Effects' "wet sidewalks cause rain" critique of how we parse information.

Still, I'm glad it's transparent in this case and thus exposable. How much more is hidden, guarded jealously even, as though methodology was trade secret instead of, you know, the underpinnings of proposed outcome?