[wenc]

> They need a journal of failed chemistry because only the working stuff gets published.

I think there's theoretical value in this, and many have tried, but the incentives/disincentives for doing so isn't favorable. Here's some reasons why I think it's difficult to motivate people to publish negative results:

1) Negative results, while important in advancing science, don't get you grants.

2) Negative results need to be peer-reviewed -- there are "good" negative results (good protocol, failed result) and "bad" negative results due to bad data collection, wrong conclusions (bad protocol, failed result).

3) Given that it's so much easier to get a negative result than a positive one (as in anything there are only few ways to be right, tons of ways to be wrong), the volume of papers to review is orders of magnitude higher. Reviewers have to really sift to find the needle in the haystack. Between teaching classes, sitting on mindless committees, writing grants, mentoring grad students, etc. academics don't have that kind of time.

4) It would incentivize poor/mediocre labs to publish a lot of negative results to get their pub count up (these are the ones that currently publish unsubstantiated positive results in fly-by-night journals).

5) Bad faith authors may publish fake negative results to throw others off a promising line of inquiry.

(Note: some of these disincentives also apply to positive results in journals today)

The current method I know for exchanging "good" negative results is word-of-mouth, usually during post-conference drinks at the bar. (works for tech too!) I'm not sure if it's possible to arrange incentives in such a way as to make publishing "good" negative results worthwhile.

EDIT: there are exceptions. If the space of solutions is known a priori and bounded (say only n ways to do something), then publishing n results if even all n are failures is worthwhile. This situation doesn't come up all the time (the solution space is often open), but when it does, it's worth publishing all n results.

[ggcdn]

Maybe instead of full-fledged journal model, negative results should be published in a short (few pages) failure report. This addresses at least a few of the points you highlight. Skip all the cruft: here’s what we tried to do, here’s our methodology, and here’s the result.

Then these could be classified by methodology/process/chemicals/etc for people to look up before starting their research.

[wenc]

Yes, sounds like that could work. Like a wikipedia of experiments tried, with contact details and a public discussion forum for anyone interested in going deeper into the results.

In many fields, you can generally tell if someone is faking it about from how they respond and what they say. It's not that easy to fake specialized knowledge. This will help increase the signal-to-noise ratio.

The trick is to bootstrap a high-quality community, where top people in the field will want to engage. MathOverflow managed to do this for pure math. Quora on the other hand used to be good in the early days (many SV names) but has struggled to maintain quality. There's also the problem of moderating professional rivalry and reputation-maintenance in academia -- these are non-trivial issues in some smaller fields.

[Vaslo]

Many really good points here I hadn’t thought much about.