[nostrademons]

I've heard that the general transformer architecture (not specifically LLMs, which imply a language model, but applied to sensory perceptions and outputting motor commands) has actually been fairly successful when applied to robotics. You want your overall assembly line to have a tiny, repeatable instruction set, but inside each of those individual instructions is oftentimes a complex motion that's very dependent upon chaotic physical realities. Think of being able to orient a part or deal with a stuck bolt, for example. AI Transformers potentially would allow us to replace several steps in the assembly that currently require human workers with robots, and that in turn makes the rest of the assembly much more reproducible (and cheaper).

Training these models takes a bunch more time, because you first need to build special hardware that allows a human to do these motions while having a computer record all the sensor inputs and outputs, and then you need to have the human do them a few thousand times, while LLMs just scrape all the content on the Internet. But it's potentially a lot more impactful, because it allows robots to impact the physical world and not just the printed word.

[grues-dinner]

And it's a nice problem to solve with AI of many kinds because you can forward-solve the kinematic solution and check for "hallucinations": collisions, exceeding acceleration limits, etc. If your solution doesn't "pass", generate another one until it does. Then grade according to "efficiency" metrics and feed it back in.

As long as you do that, the penalty for a a slop-based fuckup is just a less efficient toolpath.