[Jensson]

It has trained on countless programming tutorials out there, including bash tutorials for all kinds of things. Such tutorials often includes "create file -> ls to see file -> print content of file" etc, so GPT then takes those tutorials and creates grammatical rules how those words transform into each other. But if you start going outside of the realms of online tutorials it starts to falter quickly and then just prints nonsense.

[supermdguy]

To add to this, each token is generated by going through its network once. So there's no way computationally for it to do any reasoning or follow an algorithm. That said though it's impressive how much its able to accomplish without any procedural reasoning.

[abathur]

I've taken to saying that the ~emergent behavior here underscores what an incredible bootstrap written language is.

I think of it as a savant-esque capability. A glimpse of what we might be like if some slices of our language/memory faculties were turned up to 1111 while others are completely absent.