[johnisgood]

Of course. Even PubMed is ridden with ridiculous stuff. PubMed! To give you an example: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3068791/

Thankfully we can do our own research, see by whom it has been cited, in what journal it has been published, and so forth. You can easily tell that this is quite an unimportant and incredible one. Of course you could tell it yourself after having read it that it is nonsense.

[jwally]

Are there any guides / heuristics on how a layperson can evaluate technical literature?

For example my mother-in-law is starting to micro-dose lithium and cited this article as evidence: https://www.lifeextension.com/magazine/2020/12/lithium-and-a...

Which cites 19 medical journals. The website itself reeks of bullshit, but how can I tell if the journals it cites are valid or just woo?

[kls]

For the record, lithium as a mood stabilizer is pretty solid and settled science in the realm of mental health, which is actually saying a lot as the only other mental health medication that can really claim the same amount of success in treatment to alleviate symptoms would be amphetamines to treat attention disorders. To this day lithium is considered the gold standard in suicide preventative treatment. There is a lot of research that leads to the conclusion that due to the realities of modern civilization we may not receive enough lithium which in small concentrations is is an essential mineral. While I have not read the article you linked to, it may not be that far off base, but may very well be snake oil, nevertheless, the general scientifically held belief is that there is a strong probability that due to modern society norms we do not receive enough lithium in our diet and that the average human would benefit in metal health from a small boost in lithium in their diet. With that stated it is a good reminder that, you can take too much lithium and an lithium overdose has some nasty side effects. It is really something that should be monitored by a doctor as they take blood serum samples and ensure that you are not running high lithium levels which can destroy your kidneys and other organs.

https://www.psychologytoday.com/us/blog/evolutionary-psychia...

[deeeeplearning]

Looking at the Impact Factor of a Journal can help but it's not perfect. For example Nature and Science, generally considered the most prestigious journals in the world, have impact factors around 1000 and smaller but still quality niche Physics journals might be in the low hundreds.

The bottom line is that as a Layperson it would be extremely difficult to vet the validity of any given paper let alone whole journals.

https://www.scimagojr.com/journalrank.php

[jwally]

Bookmarking that. Thank you!

I think I was starting to draw that conclusion.

If a claim is in the Lancet, NEMJ, Nature, etc; its probably pretty credible.

If a claim is in another journal; it shouldn't have much bearing on my decision (positive or negative) since I don't have the expertise to evaluate it.

[alisonkisk]

But how do you know those are credible? It's fundamentally impossible to decide from first principles. You have to choose where to put your faith. Longstanding reputation is one popular way, but old things can sell out, or can be fairy tails (like at least all but one religions).

[deeeeplearning]

Right but at least Impact Factor gives you some kind of objective idea of the importance of a journal. Most researchers know Science and Nature are elite journals and impact factor at least backs that up by showing that their papers are highly cited by other "experts" in the field. But as I said it's still not perfect.

[pc86]

I think a more important question is whether or not a layperson should be evaluating technical literature on their own at all. I'm wholly uneducated and unprepared to evaluate even simple medical literature, so I find a doctor who I trust and who can explain why they do the things they do.

There is an idea, especially here in the US, and especially over the last 10-15 years, that you can just "do your own research" and come to correct conclusions about things that have been studied for decades or centuries. You can't. If you want to do it for your own edification, or because it's interesting, by all means. But please not for making real decisions that have a real impact.

[jwally]

Honest question. If your back hurt and your primary-care-physician recommended you see a chiropractor but you saw on wikipedia that chiropractic is pseudoscience with a risk of injury; how would you proceed?

[pc86]

This is a good question, and I'm a little biased in that I do think chiropractic is pseudoscience as medical treatment, but it also feels really good. So I go every once in a while the same way I do with a massage.

I would just tell them my concerns and either ask for why they think that would help, or maybe get a second opinion? It's hard to say for sure.

[asdff]

A good measure is to see if there is consensus in the literature. Do other papers report something similar? Is this paper widely cited and/or results accepted in more recent papers?

But to begin with, the article you cite is not a proper peer reviewed article. It's merely a webpage on a private website. It's not indexed on pubmed. No one has cited it, nor will they.

[astrange]

Except for his mother-in-law.

[jvanderbot]

examine.com

[johnisgood]

I am currently working on redesigning the entire academic publishing system which is not an easy task and since because I am alone, it may take an entire lifetime of mine, and it is on hold sadly. I wish someone could take it up with me. I was "squatting" on a domain which is about to expire because I cannot even pay for it. :/ I have no time to work on the design itself, let alone the implementation, because I have to survive on shitty money. I did work on the design quite a lot a year ago when I could afford it, but it has been a while. It would be quite easy to filter out nonsensical articles the way it would work in my case, among other features. I cannot really talk about it on here.

That said, there are lots of "may" in the article. As to why someone is telling you this, well, in this case it is to sell you their products. She could consult a doctor about it. What form is the lithium in? How much of the product (which they sell) is actual lithium? Should look at the studies mentioned where they say "may", so that you would actually have something to base your beliefs on, because "may" is not a claim of "will", and "may" is not "evidence", it is just speculation; they are merchants of hope. The article also does not go very in depth with the comparisons of dose. How much lithium do people get daily and from which sources? Might it turn out that micro-dosing is actually a mere 1/10th of what people already get normally in a day? Keep also in mind, that some regions have lithium in their tap water already. Plus, people believe the news too, because it sounds believable, but this is not about technical literature alone. People do need to get educated about how to critically comb articles.

By the way: https://pubmed.ncbi.nlm.nih.gov/11838882/

> Lithium is commonly found in drinking water and various foods, with dietary intake estimated at 0.6 to 3.1 milligrams per day in the United States in 1985. Lithium supplements are sold as pills, liquid capsules, solutions, and syrups of lithium orotate or lithium aspartate.

[jwally]

This is great info. Thanks for this.

With sites like this, it feels like the citations are a form of DDOS or EULA boilerplating where they could demand naming rights to your first born because they know you won't read it.