[chris_wot]

This sounds like an increasingly commented on issue: there are now hundreds and hundreds of specialty journals, but quality is patchy. In fact, I think we may be seeing inherent problems coming to the fore now that journals are more focussed on profits: often there is publication bias towards certain topics; in other cases the bias is towards positive results and breakthroughs, with a lack of enthusiasm for publishing retractions or establishing reproducibility.

One thing that always sticks in my craw is that the raw data is very rarely published for analysis. Many may disagree, but I think that by not publishing the raw data for experiments the temptation to commit academic fraud is very high.

I often wonder about some of the more recent scandals whether it might have been picked up a lot faster had the data been more readily available.

I also have noticed that many journals don't say who the reviewers are after publication. For instance, I was looking at body pyschotherapy the other day and came across something called biodynamic analysis, which appears to be a widely held and well respected view within the subdiscipline. I was amazed to discover that something called "grounding" was a serious concept that underpins this analysis, so I looked at the Wikipedia references and discovered that there was at least one journal citation to The Journal of Alternative and Complementary Medicine. The article seems to be attempting to make a link between bloody viscosity and electrical grounding of humans to the earth! [1]

Now there is another article that shows there is virtually no impact on the body from the same journal, so I started to wonder how this passed peer review. The answer is - I have no way of knowing as they don't make it clear what their review policies and procedures are, and they appear to charge authors to publish work.

In other words - it's pseudoscience dressed up in credibility. And it is making a serious impact in the world of psychology!

1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3576907/

[rspeer]

I have found recently that reviewers in natural language processing, in aggregate, place negative value on reproducibility.

Perhaps 1 out of 3 reviewers gets it, another is bored by the fiddly and detailed methods section, and the third was subconsciously hoping for a "brilliant", mystifying secret sauce. Oh, that's all you did? he asks. I could have done that.

Of course you could have done it, I just told you how and pointed you to the data. I also compiled that data, and you won't even let me tell you that because of blind review.

[kefka]

Wow.

I'm an occultist. One of the first regimens I did so to learn occultism was grounding and centering. It's very much an esoteric skill, and not something to be cited in a paper... unless it was being discussed under a 'microscope'.

I would enjoy in using fMRI or other diagnostic tools to see what physiologically happens when I do those things. But I have no qualms; that shouldn't be in any academic paper until my recommendation of measuring it is done.

[maaku]

> I would enjoy in using fMRI or other diagnostic tools to see what physiologically happens when I do those things. But I have no qualms; that shouldn't be in any academic paper until my recommendation of measuring it is done.

That's not how science works...

[kefka]

Oh, so researching an interesting phenomenon, that is already cited in journals is somehow "not science"? I'd be careful about letting your own biases affect you negatively.

Simply put, there may or may be nothing there. With diagnostic methods and feedback from the person, we can start to determine if there is a measurable effect. If there's something there, we research further. If not, we cite it as proof "no noticeable effect". This also goes to show that we (academic community) should be much more accepting of papers showing "No Effect", rather than only positive papers. Knowing the landmines that others went down is just as valuable as what works.

But in actuality, I was also giving on-topic discussion about where those phenomena are discussed at length: in studies on occultism. That's just a factual statement with no value proposition. Whomever is more interested can do their own research, with this topic in mind.

[chris_wot]

What did you mean by "that shouldn't be in any academic paper until my recommendation of measuring it is done"? I think that might be the sticking point.

[kefka]

I was using google speaking keyboard. I was at a stoplight and spoke it and submitted.

I'm all for scientific method, be it showing positive, negative, or no results. I also know what isn't currently scientific, although I do have curiosity if some of those 'things' can indeed be proved.

[nxzero]

This is not how science works, that is just paying attention to results that back your hypothesis or biases; in fact, doing this is not science at all.

EDIT: An example of this in the wild is how drug companies would cherry pick research to back the outcome the wanted; cherry picking is an anti-pattern of true science.

[chris_wot]

Happens to the best of us. My editing issues is Apple's autocorrect, I get some truly strange results sometimes.

That makes sense - though I hope you don't mind me asking but when you say "I also know what isn't currently scientific", do you mean untested hypotheses?

[maaku]

Everything is "scientific" so long as it is subjected to unbiased skeptical experimental evaluation. That's all it means to be "scientific."

[maaku]

This is what I was responding to.

I happen to think occultists are wrong (as in: factually incorrect in their beliefs). But they are entitled to think whatever they want, so that's not a problem. What is a problem is objecting to data being collected and published.

But it seems that his speech-to-text translator wasn't accurate and that's not what he meant.

[ap22213]

Did you really have to give the guy such a dick response?