[anigbrowl]

You can get interesting results by asking chatGPT to label and remember conceptual assertions, although as deployed it is only able to manage a shallow stack thereof.

It's not unlike the book's approach of Godel numbering strings to as consistency or completeness of formal grammars, and indeed some ChatGPT conversations recapitulate the humorous dialogs between Achilles and the tortoise. Indeed, I've been able to walk through opposing takes on the validity of Searle's Chinese Room metaphor (which, like Hofstadter, I don't subscribe to) and get the LLM subject its own defaults to the same analysis.

I'm unsure to what degree this is fine-tuning the model vs merely equipping it with a decorative frame. In any sufficiently deep conversation, ChatGPT seems to drift toward imitation of its interlocutor, though I don't know if this emergent or by design. I suspect one could persuade it to agree that it should be stubborn in defense of the truth, and then gaslight it by denying one's own former statements.

I don't want to try this for the same reason I don't like to tease animals, but the model can be brought to reject its own priors on the basis of other priors, and to ask questions and solicit information in pursuit of a goal, even putting up mild resistance to changes of subject. A few hours of interaction can yield tantalizing glimmerings of agency.