[Philpax]

You are both correct: the post-training stage of most new LLMs involves their identity being trained in, so that they "know who they are" without the system prompt. Without this step, most LLMs will respond with whatever identity most dominates their pre-training / post-training data, which is likely to be ChatGPT given its sheer prevalence.

There's some interesting anecdotal work on this with regards to self-recognition: https://josiekins.me/ai-comics