[namelos]

> It's still just predicting the next word.

Computer-generated random numbers are not truly random, yet they are practically random in most real-world use cases. You can’t easily cheat the RNG in World of Warcraft to get critical strike every time.

The output from GPT is generally very intelligent and versatile in terms of text. It may even be capable of handling more multi-modal problems with the use of enough sensors and motors. Perhaps the same idea of "predicting the next move" or "predicting the next idea" can still apply.

Who knows, maybe humans are essentially physical creatures that "generate the next thought and generate the next move"?

One of the biggest issues with GPT is its lack of mid-term memory like human do. Instead, we need vector store and search then bolt back its short term memory instead of letting it handle everything in a more coherent way. Perhaps it could benefit from lightweight fine-tuning technologies like LoRA and hypernetworks for stable diffusion. If this issue is resolved we would see it'll get even more practical. Again, the flaw is not about "predicting the next words".