Meanwhile, 11 hours ago over on Reddit a dude has connected a robot to ChatGPT. The robot sees and talks, and clearly has motorized limbs. All you need now is to turn verbs into servo commands:
Cute, there are a number of people and labs working on integrating GPT models with robotics. I think you'd need to train a model on prompt-action pairs to get the best results but you could potentially do what they did in the Voyager paper https://arxiv.org/abs/2305.16291 and have a pre-trained LLM just write the raw programs to control the robot and then save and re-use them with function calling.
https://www.microsoft.com/en-us/research/group/autonomous-sy...
https://www.microsoft.com/en-us/research/group/applied-robot...
https://blogs.nvidia.com/blog/eureka-robotics-research/
https://www.vice.com/en/article/93kb43/princeton-uses-gpt-to...
https://techcrunch.com/2023/11/10/ai-robotics-gpt-moment-is-...