[djaque]

That's kind of my experience with research, but in physics. Already have three projects under my belt w/ no publishable results. All of them involved much more work than the other grad. students in my lab including building like three different instruments from scratch, but the other students have published. Their projects were much more "safe" than mine and their papers are pretty small, but at least they have papers.

[joshvm]

Don't worry. Instrumentation is a pain for publishing because often it creeps into engineering territory. I spent a year of my PhD trying to get something to work, it didn't and I ended up building an entirely new instrument in the third year. That's how it goes. I know people who came out of their PhDs with five papers (in the UK that's a big deal), but they're in fields where its relatively easy to publish and/or got lucky with their topic. Or you work with someone with a funky new dataset that breeds publications.

I think most instrumentation scientists are sympathetic to this. There are also lots of instrumentation jobs that don't require stupid numbers of publications because it's not practical to find candidates. There are relatively few good hardware people in the sciences (especially fields where you can't get a mech/electrical engineer).

I suggest finding some conferences to start. They're a good venue for telling people what you did. There are also journals specifically for building stuff, SPIE has a lot for astrophysics, for example.

[djaque]

Thanks, that's good to hear. Fortunately I do have two papers in the works right now, one of them from an instrument I made and another from some simulation work.

The engineering really is the crux of the problem. I personally find it very interesting and I feel like an oddity in my department for that. Most other physicists kind of look down on the "e-word" to the point that my advisors made me change references to "engineering challenges" into "experimental challenges" for a conference talk I gave. I'm in a very equipment heavy field too which is why I find it so strange.

What's frustrating though is that sometimes it feels like I'm only doing engineering with no physics in the mix. Hopefully thay will change over the next year though.

[cushychicken]

What field would you go into if you were interested in making instruments for scientific research? Would that be engineering physics?

[joshvm]

Instrumentation is field-specific. It really depends how bespoke your requirements are. Some places buy stuff off the shelf, others fabricate in-house.

In astronomy it's called instrumentation. Astro has relatively sane nomenclature. You have observational (looking at stuff and gathering data), theoretical (theory and simulations) and instrumentation (building stuff). Everyone in the field understands those terms. That said, instrumentation spans everything from construction, design, calibration/characterisation, etc.

In other branches, as the OP pointed out it may be "experimental" physics. Problem with that term is it's extremely vague. That's essentially anything which doesn't involve theory or simulation - lab work. That doesn't necessarily mean you actually build anything though. I was surprised at how little we got taught about instrumentation during my physics degree. Of course we had labs, but it was more about using kit than how it was built. It's almost as if it's someone else's problem, even though that's not how research actually works.

The irony of physicists sneering at engineers is that most labs employ a bunch of them who do most of the actual design and build work. They tend to be less focused on publication, but they are absolutely critical employees. The sorts of people who've spent 20-30 years working on some uber niche detector tech and know more than the people writing the papers for sure. A good chunk have physics PhDs.

When you get to big money stuff like particle accelerators and telescopes, a lot of this is contracted out. Physicists and engineers are responsible for designing the spec, but it's less tinkering in a lab with hardware. In industry it's mostly called engineering (e.g. optical, mechanical, electrical), but I've seen engineering physics too. That tends to be in companies that build very niche equipment specifically for physics research.

[djaque]

Exactly, the instruments I was talking about are used to measure the properties of photocathodes. It's such a niche field that there's no off the shelf solution. Everybody makes their own device and ours is one of like two in the world with it's grade of sensitivity.

There is one professional engineer in the lab my group is a part of, but they primarily work on the bigger projects. For this, I have to do all of the CAD work and talking to the machinists. I also machined a lot of the parts myself due to time constraints.

[Balgair]

I'll echo this too. Instrumentation is field specific. I was in bioeng, but my work was photo-chem/physics and optics. Basically, building novel nanoscopes.

Pro-Tip: Want a 'quick' nobel? Do optics for ~3/4 years. Then never touch it again. Making new kinds of microscopes is crazy useful and high impact, but you either get lucky or you waste 45 years in a lab. Try it out for a bit, throw a few on red, then walk away from the table.

[djaque]

I'm in accelerator physics. Basically we study the physics of charged particle beams with the goal of improving their properties/developing new applications. I thought it was the perfect combination of engineering, which I like, with pure physics.

[wodenokoto]

I never got published in the field I have a masters in, for what I'd like to think of as somewhat comparable reasons.

However, my take-away was that the successful researchers were the ones who could take any decent experiment and figure out what was publishable about it, or at least steer it into a publishable direction.

[dguest]

What branch of physics? There are hundreds of published null results looking for dark matter or Supersymmetry, for example. Instead of documenting a failed search, the papers focus on what versions of the theories the experiment excludes to 95% confidence.

[djaque]

I'm in acceleraror physics and study photocathodes. Unfortunately the state of those projects is that we thought a material should have great photoemission properties under some very specific circumstances.

When we measured it, the material was actually terrible and we don't understand why it's that way outside of some speculation. Unfortunately it also feels like our lab doesn't have the type of expertise to explain why it's so bad either at least not without years of learning.

My advisor's explanation on publishing was that if we had great results that followed our initial theory of what's going on then we could write a paper and nobody would question it. But, now that we contradict theory the threshold of evidence is much higher and he doesn't want to go ahead with it.

We trust our data, but he's concerned about getting it through peer review. For what it's worth we did publish it in a conference proceeding, but it doesn't have the same weight as a journal article.

[darawk]

> My advisor's explanation on publishing was that if we had great results that followed our initial theory of what's going on then we could write a paper and nobody would question it. But, now that we contradict theory the threshold of evidence is much higher and he doesn't want to go ahead with it.

Isn't that sort of the most important thing to publish? Something that contradicts theory means something new. Seems like all the more reason to pursue it, no?

[missosoup]

In a vacuum yes. In the world of academia and its perverse incentives, no.

Publishing something that goes against the grain means a lot more scrutiny on your work and bigger humiliation for any mistakes made. If you're publishing against the grain, you have to double triple check every single detail to make sure it's all iron-clad. This costs a lot of additional time and money. When publishing research that preaches to the choir, no such considerations are required.

In an ideal world, publishing against the grain should be encouraged and there should be no extra reputational penalty for getting caught with errors in that type of research (heaps of errors go unnoticed in mainstream papers, even popular ones). But that's not how it works sadly.

[jschwartzi]

Really academia needs a way to request comment on an experimental procedure without requiring results or conclusions. Then you could publish negative or questionable results and seek speculation from other researchers.

[ridaj]

It does: for example, as djaque's comment points out, in their case this was published at conference proceeding. And of course advice can be sought in private as well.

[vageli]

> Really academia needs a way to request comment on an experimental procedure without requiring results or conclusions. Then you could publish negative or questionable results and seek speculation from other researchers.

I feel like anonymity could possibly go a long way for this.

[ridaj]

It reminds me of the paper reporting that neutrinos had been detected going faster than the speed of light[0]. It turned out to be an instrumentation issue. Some of the folks involved had to step down.

Yes, it is important to publish results that violate established theories, but if there's already a lot of validation of those theories through experimentation, you're going to have legitimate pushback, especially if your method of experimentation isn't the most sophisticated.

If the result of your publication is a theoretical breakthrough, then that's great, but unless you're in the top tier within that discipline, the likely outcome is that your research turns out to have a big flaw which you missed, and you invite a bunch of people to point out to the world that you can't do science properly...

[0] https://www.sciencemag.org/news/2012/06/once-again-physicist...

[djaque]

Thanks, this is in line with what I feel is going on. I woke up this morning to see a lot of comments on how my experience is proof that academia is broken. Instead, it just feels like everybody is being very cautious about our results.

We don't understand why our data doesn't line up with theory and this isn't some super well funded experiment like the ones at the LHC with like a million grad. students. This is just me in the lab alone with the hundreds of subtle ways the experiment could have gone wrong or needed a slight modification to the theory. It would take us a lot of effort to distinguish an experiment gone wrong from an interesting new development and that's not always worth it. Especially for such a niche area of research like I'm in.

Also, we did publish our data. We just did it in a conference proceeding with a lot of caveats attached to it instead of in a big journal article. That's the aspect that I was complaining about, but I do understand my advisor's decision.

I have to strongly disagree that science is broken here.

[michaelt]

If the theory says it's possible to juggle five chainsaws but nobody's managed to demonstrate it, and I try it without success, have I proven it impossible? Or have I just proven I'm not good a enough juggler?

[djaque]

Exactly, that's the state of the experiment. There's just so many things that could have gone wrong without us noticing. Even though we trust the instrument is working, we don't know if the material under study is exactly as we expect.

[BurningFrog]

Yet another example of how 2020 institutional science is broken.

For the purpose of producing knowledge. It does other things.

[djaque]

Could you please explain why you think this means science is broken?

There are real issues in academia, but I disagree that what happened here is wrong. We did publish our data, we just didn't do it in a journal article. In my fields, this is actually fairly typical and journals are less of the norm unless you have a big result.

The whole problem is that extraordinary claims require extraordinary evidence. Sometimes when you don't have much funding and there's only a few grad. students in the lab, that evidence is no longer worth the time to collect.

See the comments below about the "neutrinos travel faster than the speed of light" paper. There is a real danger in publishing theory-breaking results when you don't have 100% confidence in them.

[BurningFrog]

> Could you please explain why you think this means science is broken?

To be precise, I already think science is broken, and your story seemed to confirm it.

Obviously, that "this is actually fairly typical" is not a counterargument to a systemic belief.

That said, it's entirely possible I jumped the gun here.

I take the reproducibility crisis to be a strong indicator that something is Very Wrong with science. I'm also convinced that the modern peer review system is fundamentally broken.

[101404]

Why would the results not be publishable?