[darawk]

> My advisor's explanation on publishing was that if we had great results that followed our initial theory of what's going on then we could write a paper and nobody would question it. But, now that we contradict theory the threshold of evidence is much higher and he doesn't want to go ahead with it.

Isn't that sort of the most important thing to publish? Something that contradicts theory means something new. Seems like all the more reason to pursue it, no?

[missosoup]

In a vacuum yes. In the world of academia and its perverse incentives, no.

Publishing something that goes against the grain means a lot more scrutiny on your work and bigger humiliation for any mistakes made. If you're publishing against the grain, you have to double triple check every single detail to make sure it's all iron-clad. This costs a lot of additional time and money. When publishing research that preaches to the choir, no such considerations are required.

In an ideal world, publishing against the grain should be encouraged and there should be no extra reputational penalty for getting caught with errors in that type of research (heaps of errors go unnoticed in mainstream papers, even popular ones). But that's not how it works sadly.

[jschwartzi]

Really academia needs a way to request comment on an experimental procedure without requiring results or conclusions. Then you could publish negative or questionable results and seek speculation from other researchers.

[ridaj]

It does: for example, as djaque's comment points out, in their case this was published at conference proceeding. And of course advice can be sought in private as well.

[djaque]

Yes, exactly. I also gave an invited talk about it at the largest domestic conference in my field. This is kind of unusual for a grad. student which should give you an idea of how unique the problem is. We actually received good advice about our experiment that we might try later. The issue is just how niche what I study is. There's probably less than 10 experts in the world on the specific problem I was tackling.

[vageli]

> Really academia needs a way to request comment on an experimental procedure without requiring results or conclusions. Then you could publish negative or questionable results and seek speculation from other researchers.

I feel like anonymity could possibly go a long way for this.

[ridaj]

It reminds me of the paper reporting that neutrinos had been detected going faster than the speed of light[0]. It turned out to be an instrumentation issue. Some of the folks involved had to step down.

Yes, it is important to publish results that violate established theories, but if there's already a lot of validation of those theories through experimentation, you're going to have legitimate pushback, especially if your method of experimentation isn't the most sophisticated.

If the result of your publication is a theoretical breakthrough, then that's great, but unless you're in the top tier within that discipline, the likely outcome is that your research turns out to have a big flaw which you missed, and you invite a bunch of people to point out to the world that you can't do science properly...

[0] https://www.sciencemag.org/news/2012/06/once-again-physicist...

[djaque]

Thanks, this is in line with what I feel is going on. I woke up this morning to see a lot of comments on how my experience is proof that academia is broken. Instead, it just feels like everybody is being very cautious about our results.

We don't understand why our data doesn't line up with theory and this isn't some super well funded experiment like the ones at the LHC with like a million grad. students. This is just me in the lab alone with the hundreds of subtle ways the experiment could have gone wrong or needed a slight modification to the theory. It would take us a lot of effort to distinguish an experiment gone wrong from an interesting new development and that's not always worth it. Especially for such a niche area of research like I'm in.

Also, we did publish our data. We just did it in a conference proceeding with a lot of caveats attached to it instead of in a big journal article. That's the aspect that I was complaining about, but I do understand my advisor's decision.

I have to strongly disagree that science is broken here.

[michaelt]

If the theory says it's possible to juggle five chainsaws but nobody's managed to demonstrate it, and I try it without success, have I proven it impossible? Or have I just proven I'm not good a enough juggler?

[djaque]

Exactly, that's the state of the experiment. There's just so many things that could have gone wrong without us noticing. Even though we trust the instrument is working, we don't know if the material under study is exactly as we expect.