[ricopags]

Google's leading here at the moment. Check out their work on SayCan[0], RT-1[1], and Language Table[2,3]

[0]https://ai.googleblog.com/2022/08/towards-helpful-robots-gro...

[1]https://ai.googleblog.com/2022/12/rt-1-robotics-transformer-...

[2]https://ai.googleblog.com/2022/12/talking-to-robots-in-real-...

[3]https://github.com/google-research/language-table