[gryfft]

Just for fun, I reproduced this in GPT-4o, then continued to challenge ChatGPT to notice the flaw in its reasoning. It finally did when I asked why it was excluding Alice from her brother's sisters.

I asked it to present a fact which, if remembered, would help it not make the same mistake. Its first idea, "When considering sibling relationships in a family, always account for the person in question as part of the sibling count unless specifically excluded," was too specific, so I asked it to generalize this insight.

Its response: "When analyzing a problem, always ensure to fully account for all relevant entities and their relationships. Double-check your assumptions and consider the problem from multiple perspectives to ensure completeness and accuracy."

I told it to commit this fact to memory and generated a fresh session with the original AIW prompt. It then answered successfully.

I would like to find some other prompts like the AIW prompt that LLMs struggle with and test the effect of this "remembered fact" on them.