[setgree]

I asked

> Can you find an academic article that _looks_ legitimate -- looks like a real journal, by researchers with what look like real academic affiliations, has been cited hundreds or thousands of times -- but is obviously nonsense, e.g. has glaring typos in the abstract, is clearly garbled or nonsensical?

It pointed me to a bunch of hoaxes. I clarified:

> no, I'm not looking for a hoax, or a deliberate comment on the situation. I'm looking for something that drives home the point that a lot of academic papers that look legit are actually meaningless but, as far as we can tell, are sincere

It provided https://www.sciencedirect.com/science/article/pii/S246802302....

Close, but that's been retracted. So I asked for "something that looks like it's been translated from another language to english very badly and has no actual content? And don't forget the cited many times criteria. " And finally it told me that the thing I'm looking for probably doesn't exist.

For my tastes telling me "no" instead of hallucinating an answer is a real breakthrough.

[Wowfunhappy]

> For my tastes telling me "no" instead of hallucinating an answer is a real breakthrough.

It's all anecdata--I'm convinced anecdata is the least bad way to evaluate these models, benchmarks don't work--but this is the behavior I've come to expect from earlier Claude models as well, especially after several back and forth passes where you rejected the initial answers. I don't think it's new.

[GoatInGrey]

I can concur that previous models would say "No, that isn't possible" or "No, that doesn't exist". There was one time where I asked it to update a Go module from version X.XX to version X.YY and it would refuse to do so because version X.YY "didn't exist". This back with 3.7 if I recall, and to be clear, that version was released before its knowledge cut off.

I wish I remembered the exact versions involved. I mostly just recall how pissed I was that it was fighting me on changing a single line in my go.mod.

[setgree]

alas, 4.5 often hallucinates academic papers or creates false quotes. I think it's better at knowing that coding answers have deterministic output and being firm there.

[esafak]

It would have to have been trained on the papers without being aware of retractions for that test to work. Otherwise it will be limited to whatever papers it gets from a search engine query, which likely won't contain any un-retracted illegitimate papers.

[setgree]

I once worked at a crypto company that outsourced some of its marketing work to a content marketing firm. A piece that firm submitted to us contained a link to an "academic" article about global poverty with a totally garbled abstract and absolutely no content whatsoever. I don't know how they found it, because when I search google scholar for a subject, usually the things that come back aren't so blatantly FUBAR. I was hoping Claude could help me find something like that for a point I was making in a blogpost about BS in scientific literature (https://regressiontothemeat.substack.com/p/how-i-read-studie...).

The articles it provided where the AI prompts were left in the text were definitely in the right ballpark, although I do wonder if chatbots mean, going forward, we'll see fewer errors in the "WTF are you even talking about" category which, I must say, were typically funnier and more interesting than just the generic blather of "what a great point. It's not X -- it's Y."

[itissid]

When Claude does WebSearch it can delegate it to a sub agent which of it ran in the background will write the entire prompt on a local file and the results. If that happened, I would like to know what it gave you for that. It is always very interesting to know the underlying "recall" of such things. Because often it's garbage in garbage out.

The location might still be on your disk if you can pull up the original Claude JSOn and put it through some `jq` and see what pages it went through to give you and what it did.

[setgree]

this is via the chat, but it did provide this

> The user wants a real, sincere academic paper that looks legitimate — proper journal, real authors with real affiliations, many citations — but is obviously nonsense or garbled. Not a hoax or deliberate prank, but something that was genuinely submitted and published, yet is clearly meaningless if you actually read it. Let me think about what I know and search for examples. There are some famous cases:

> Papers generated by AI/GPT that slipped through - there have been reports of papers with phrases like "Certainly, here is a possible introduction for your topic" or "As an AI language model..." > There have been papers with obvious tortured phrases - "sham" papers that use synonym substitution to avoid plagiarism detection, producing phrases like "sheep pursuing" instead of "shepherd" or "big data" becoming "large information" I'm also thinking of papers that made it through peer review despite being complete gibberish, like the famous SCIgen computer-generated papers that fooled conferences, or papers where the abstract and content don't match at all.

> Now I'm trying to recall specific real examples that fit this criteria — published papers that are genuinely incoherent or nonsensical but appeared in legitimate venues.

This is maybe not the _exhaustive_ search strategy I would have liked, but it's probably something you would do if you were limited to just a certain amount of effort on a task.

[lgas]

Well, if there are papers that match your criteria, it's hallucinating the "no".

[Jimmc414]

It might be wrong but that’s not really a hallucination.

Edit: to give you the benefit of doubt, it probably depends on whether the answer was a definitive “this does not exist” or “I couldn’t find it and it may not exist”

[setgree]

claude said "I want to be straight with you: after extensive searching, I don't think the exact thing you're describing — a single paper that is obviously garbled/badly translated nonsense with no actual content, yet has accumulated hundreds or thousands of citations — exists as a famous, easily linkable example."

[psychoslave]

That's still less leaned toward blatant lies like "yes, here is a list" and a doomacroll size of garbage litany.

Actually "no, this is not something within the known corpus of this LLM, or the policy of its owners prevent to disclose it" would be one of the most acceptable answer that could be delivered, which should cover most cases in honest reply.

[terminalshort]

And there are: https://en.wikipedia.org/wiki/Sokal_affair

[cgh]

> no, I'm not looking for a hoax, or a deliberate comment on the situation. I'm looking for something that drives home the point that a lot of academic papers that look legit are actually meaningless but, as far as we can tell, are sincere

The Sokal paper was a hoax so it doesn’t meet the criteria.

[terminalshort]

The fact that it got published means there is at least one whole journal full of that

[matchagaucho]

Results from a one-shot approach quickly converge on the default “none found” outcome when reasoning isn’t grounded in a paper corpus via proper RAG tooling.

[gertlex]

Can you provide more context to your statement? Are you talking about models in general? Or specific recent models? I'm assuming "one-shot approach" is how you classify the parent comment's question (and subsequent refined versions of it).

[matchagaucho]

Large models in general. A semantic query for "fake articles", without examples, is a wildcard search.

A commercial RAG solution would use Query Expansion (QE) and examples to find nearest neighbors.