[mgraczyk]

The reason why it isn't useful to ascribe intentions without a mechanistic explanation of intentionality is because you will incorrectly predict what the model will do in surprising ways.

I think it's true that current generation LLMs could, in principle, have intentionality in the way described in the article. But they would have to be trained on many orders of magnitude more data than current models.

Also AutoGPT does not work. I encourage the author to play around with it and try to get it to do something useful with high success probability.