[aDyslecticCrow]

Yes, if you want an LLM that doesn't listen to instructions and just endlessly babbles about anything and everything.

What turned GPT into chatGPT was a lot of structured training with human feedback.

[rahimnathwani]

Exactly. Section 4.3.7 briefly explains how they trained the model to better follow instructions ('steerability').