[avip]

Sure. I see 3 recurring issues with "nutritional research" I'm exposed to:

1. Wayyyyy too much industry involvement, much of it not properly disclosed (OP here is a classical and representative example).

2. Lots of research from relatively small and "less known" institutes, done by researchers with hard to verify credentials.

3. Low scientific standards. That'd include:

  - Refusal to share data
  - Refusal to share data analysis methods
  - Not keeping the basics of a double-blinded test
  - No reproduction, and not even the possibility of independent reproducibility, as the methods are described so vaguely.

That's in a nutshell.

[nonbel]

This sounds like standard biomed to me:

  - Refusal to share data
  - Refusal to share data analysis methods
  - Not keeping the basics of a double-blinded test
  - No reproduction, and not even the possibility of independent reproducibility, as the methods are described so vaguely.

Can you link to a recent paper that doesnt run afoul of at least one of your criteria. Especially the last...

[ianai]

You identified the problem. What’s the solution?

[neuronic]

The solution is identification of bad methods and subsequent ignorance.

If only we had a metric for credible science that takes authors, institutions, publisher and funding sources into account....

[betageek]

This seems like a great idea - is there some reason this doesn't exist?

[ianai]

Something that takes only authors, publishers, and institutions into account sounds counterproductive.

I think it’d be better to have an agreement that only reporducible papers are credible. Ie data the paper is reported on (to include “cleaned” data) and any tools developed (software tools) must always be included. As well as funding for everything disclosed.

Ultimately though I think we may just be seeing the obsolescence of current statistical theory. Science needs hypotheses to be testable in “trustable” metrics. But there are too many loop holes if things like p-hacking are possible.