[corty]

Actually, asking for the credibility of a journal is the wrong question. The real one is about whether an article is credible. But the answer to that is disappointingly complicated.

Articles are peer-reviewed for their readability (for peers, not laymen), consistency (e.g. if there is a mathematical proof in there, the reviewer might check it if possible. if there is an experimental technique, the reviewer might check if that technique could produce the results claimed) and reproducability (is the experiment described in sufficient detail? does the software provided produce the expected results? do the provided numerical results fit the claims?). All those checks are what the reviewer can do in a few hours, days at most. That is what peer-review means.

All the things to check beyond those are left as an exercise to other scientists after publication. Reproduction is a big task, usually almost as big as the original paper. Proofs are often extremely hard to check, so a few days by any old reviewer won't cut it. So if you want to know if the results in a paper are true, watch literature for the following years and look at publications citing that paper and (dis-)agreeing with its results. Same for (sometimes) letters to the editor, retractions and "everybody-knows"-rumours.

More reputable journals tend to attract better-quality papers (according to the peer-review criteria outlined above). That tends to correlate with a higher probability of the paper being true. But it is not a very strong correlation, there is utter nonsense, groupthink and polished turds even in very high profile journals.

[bonoboTP]

Right, I think many people believe peer review to be some big expert committee giving a paper lots of analysis and evaluation. In reality they are mostly PhD students with 1-5 years experience, not experts with decades of experience and one person may get like 5-10 papers to review at a time. Oh and it's unpaid work that gets time away from your own research and is not really incentivized beyond a vague sense of moral duty towards the progress of mankind's knowledge. The end result is that it's mostly pattern matching, does it look like the typical paper in the field? The gut reaction and impression strongly influences the decision, then the actual review is about justifying that decision.

I mean it's not totally arbitrary, really good reviewers do give it like 2-5 hours, but it's best thought of as a rudimentary filter rather than a meticulous verification.