|
|
|
|
|
|
by visarga
306 days ago
|
|
|
Verification is the bottleneck, not ideation. LLMs can generate anything on tap, but solving any non-trivial problem requires iteration between thinking, doing and observing outcomes. The real world is too complex to be simulated by AI or humans. The scientific method works the same way, we are not exempt from having to validate our ideas. But as humans we have better feedback and access to context and we can assume risks on our own. AI has no skin and bears no responsibility. So the missing ingredient for AI is access to environment for feedback learning. It has little to do with AI architecture or datasets. I think a huge source of such data is our human-LLM chat logs. We act as LLM eyes, hands and feed on the ground. We carry the tacit knowledge and social context. OpenAI reports billions of tasks per day, probably trillions of tokens of interactive language combining human, AI and feedback from the environment. Maybe this is how AI can inch towards learning how to solve real world problems, it is part of the loop of problem solving, and benefits from having this data for training. |
|
|
In my use of cursor as a coding assistant, this is the primary problem. The code is 90% on the mark, but still buggy, and needs verification, and the feedback it gets from me is not with full fidelity as something is lost in translation.
But, a bigger issue is that AI has only some solution templates for problems that it is trained on, and being able to generate new templates is beyond its capability as that requires training on datasets of higher levels of abstration.