[BoorishBears]

It's trivial to prove this is wrong: invert relationships it knows about and it fails to answer based on knowledge it previously demonstrated (even with loads of hints)

https://chatgpt.com/share/680dc86c-f0dc-800d-9f04-57ba2f126a...

https://chatgpt.com/share/680dc90b-de28-800d-92b6-f2ef824777...

Note how applying increasing pressure to answer was what caused the hallucination: hallucinations aren't tied to if the model "knows" something.

Once the tokens output don't fall into the start of some varation of "I don't know", the model is going to answer regardless of what it knows.

[koakuma-chan]

This is a logical problem that requires reasoning. Bring the fact that Barry White’s mother is Sadie Marie Carter into the context and it should then be able to infer who Sadie Marie Carter’s son is.

[BoorishBears]

You're saying it requires reasoning, then suggesting a non-reasoning based solution (steering the model in-context)

Reasoning doesn't solve it. O3 thinks and comes up empty, and O4-mini thinks and then hallucinates much worse than even 4o did: https://chatgpt.com/share/680e7a5c-ae3c-8004-bff9-5c8a0215a1...

[koakuma-chan]

The model only knows who the mother of Barry White is. It doesn’t know who the son of Sadie Marie Carter is. By reasoning I meant that it could figure out who the son of Sadie Marie Carter is if you brought into the context that Sadie Marie Carter is the mother of Barry White beforehand.