[strken]

Perhaps Gell-Mann moved from a paper that was bad at writing about physics but good at writing about Palestine, to a paper that was good at writing about physics but bad at writing about Palestine. This kind of transition seems like it might help maintain a filter bubble.

[Bartweiss]

This is certainly a risk, but unfortunately we can extend the argument to suggest that you can't ever trust any source. Say Gell-Mann wants to find a paper he can trust on Palestine. He's good at physics, and checks reporters covering both topics.

A reporter might be good or bad at covering each topic, which leaves us four cases to consider. If they're bad at physics, they could be a specialist in Palestine, but they could also be generally incompetent; Gell-Mann has no way to tell. If they're good at physics, they could just be talented reporters, but they could also be physics specialists who don't know Palestine any better than Gell-Mann does. And once again, he has no way to tell.

Even probabilistically, we could argue for either approach. Most people aren't experts in more than one thing, and it's easier for an expert than a random fool to garner attention for a baseless claim, so perhaps we should especially distrust good physics writers on Palestine. But incompetence is broadly correlated, and journalism skills apply to both topics, so perhaps we should view bad physics writing as a sign of weak fundamentals and distrust it everywhere.

(I'm talking about reporters instead of papers, but we can push the argument back a level easily; editors have to hire reporters for fields they don't know.)

Going alone, all I can see to do is to look for people who claim to specialize in a few things, one of which you know well, and trust them on their other specialties. As a society, we can perhaps do better by asking a bunch of experts who's competent in their domain and looking for alignment - provided we can all correctly agree on some experts in advance.

edit: a bit of searching suggests this is basically Berkson's Paradox. If we (boldly) assume that news sources which are bad on all topics don't circulate, then quality in one area lowers the expectation of quality in other areas.

[ddulaney]

Although there is another interpretation (especially in traditional newspapers): you aren’t just evaluating the the individual journalist, you’re evaluating the editorial staff. They are responsible for finding a physics expert to write about physics and a Palestine expert to write about Palestine.

If they do a poor job of selecting a physics expert, then it seems likely that they will do a poor job of selecting other kinds of experts as well.