Hacker News new | ask | show | jobs
by ur-whale 2856 days ago
If you're embarrassed by your research code, how do you know it works, and how do you know what you're going to publish isn't built on quicksand?
2 comments

Embarrassed by its cobbled-togetherness. Not in whether or not it works.
Not really answering the question, which was "how do you know it works?"

I would contend that if you have proven to yourself that your code works and are therefore capable of proving it to other folks (via e.g. solid testing), you should not be ashamed of the spit and glue.

It is research code, we all know what that looks like. But if, on the other hands you haven't proven to yourself it works, then it's definitely something to be ashamed of - scientifically speaking.

It’s much the same reason why you wouldn’t go to work wearing a days old, stained shirt and ripped pants — yeah, you’re more than likely going to work just as hard and as well than if you were wearing clean clothes, but that’s not going to stop people from judging you based on your appearance.

Most academics write code to just work. Not work well, or to be generalized, or to be efficient, just work. And while that’s absolutely fine, as your results being reproducible from the code is all that really matters, a lot of people don’t see it that way and will only see code slapped together haphazardly and dismiss you because of it.

This article shows very clearly why that does not work. The Princeton team could not reproduce the Berkeley results, yet the inaccessibility of the code meant that the latter persisted as a road-block for almost a decade.

Imagine if this reproducibility excuse were applied to experimental results and technique: we don't have be careful or explain in detail what we are doing, as reproducibility will take care of any errors. One consequence would be that, as the current state of knowledge became less certain, it would become less clear what to do next.

Wearing dirty clothes to work doesn't end up wasting 7 years of other people's time.

Publishing scientific papers that no one can re-implement does, and hugely so.

Therefore, not a very adequate comparison.

"Maths" is how you know it works.

Especially in fields like physics there are many limits that you can derive analytically. Some times those are highly non-trivial.

I understand the embarrassment, I am sometimes embarrassed as well looking at quickly-put-together code

I do appreciate papers that give you either a reference implementation or at least enough details on how it was done. I spent too many hours on papers that didn't trying to reproduce their claimed results...

Usually testing. Just because code works flawlessly doesn't mean it isn't embarrassing; and just because you are proud of the code doesn't mean it's bug free.