[js8]

If I add a small condition that makes the solution impossible, will it recognize that? Will it recognize for your example that it's a variation? Will it still be able to solve it when it is just a subtask of a bigger input?

If I ask it a leading question that intentionally relies on a wrong solution, will it recognize that?

[ofrzeta]

I can't answer your questions (yet) but I have an observation. In the left sidebar where it summarizes the conversations in its "own words" the canonical example gets tagged with "River Crossing Puzzle" while my variation with the chasm gets tagged as "Transporting animals across chasm".

In the canonical example it also remarks "This is a classic river crossing puzzle" before delivering the solution.

EDIT: I tried some variations with "two wolves and a rabbit" and "three wolves and a rabbit". ChatGPT started bullshitting about its solution that supposedly was "a bit more complex". It started with taking one wolf to the other side. After I pointed out the rabbit would be eaten by the remaining two wolves it apologized for the mistake but just kept going hallucinating "correct" solutions.