[programnature]

Do you think the world would be better off if Mathematica, Matlab, SPSS etc didn't exist?

In reality, they make experiments more repeatable, not less. The real offender is the in-house, proprietary software developed by individual research groups. It is almost never open sourced. And it is far more likely to be riddled with bugs.

Computer experiments are just that: experiments. Any real researcher employs multiple methods to confirm their results.

[mayanksinghal]

> Do you think the world would be better off if Mathematica, Matlab, SPSS etc didn't exist?

No, but I will celebrate a decent open source alternative, which is what the root was probably suggesting as well. Mathematica is great as an entry software - much like MS Word for word processing. But having LibreOffice is good.

[programnature]

I'm all for open source.

I just don't think the reproducibility of experiments is a valid argument against commercial scientific software.

It's an age-old purist argument. In the meantime, people have been getting stuff done. And without those tools, the "huge problem" would be even huge-er.

The real problem isn't reproducibility, it is extensibility. The development agenda isn't under your control. So if you get to the edge of a field, you might find you hit a wall.

[mayanksinghal]

> The real problem isn't reproducibility, it is extensibility

I think its both. Extensibility is obviously an issue. But so is extensibility, I will give two reasons for it:

1. Easy reproducibility is necessary for extensibility. Firstly, academia is not very good at publishing their tools or their codebases. We have given so much weight to the concept behind the implementations and not the implementations themselves, that most people skip publishing implementations. What it means is that the next research group now has to start from scratch in implementing the concepts before they can think of extending the work. Reproducibility is not only to verify previously reported results, but also to create a starting point for further work. Secondly, given that the tools that the researcher is using is proprietary, the trend is to make it closed source. It may be because the tool is not ubiquitous and hence the researcher sees no point in distributing his/her implementations - or because he had not followed any guidelines (or in case of Matlab and Mathematica - they didn't exist/were-not-popular). He might not be sure about his implementations, and hence cannot publish them.

2. Reproducibility has always been the base for science. I don't need to trust the work a random researcher that I don't personally know. I can just verify his/her findings myself. The requirement of commercial software creates a huge monetary barrier in this. It is wasteful of me to buy a licence for a simple verification that I am not planning to extend. Given that non-academic licenses of most of these softwares are insanely expensive, it makes this verification to be confined to researchers from big research groups in large companies.