[krackers]

I wasn't too surprised by the poems and song outputs since those don't seem much more novel than previous SOTA, but OPs example of chatGPT being able to effectively simulate itself is really mindblowing. It means that "being able to model yourself and recursively reason about yourself" isn't a valid claim to distinguish human consciousness from whatever this network is doing anymore.

See also https://en.wikipedia.org/wiki/Attention_schema_theory (unrelated to the notion of "attention" used in ML)

[uptheroots]

Makes me think of Godel, Escher, Bach. It's been a while since I've read it but I recall Hofstader making the argument that self-reference is the key element of consciousness.