[netcan]

Like both will be done. Idk what the roi is on adding video data to the text models, but it's presumably lower than text.

There are just a lot of avenues to try at this point.

[llSourcell]

no its not lower than text, its higher ROI than text for understanding the physics of the world, which is exactly what videos are better at than text when it comes to training data

[AstroBen]

Does that transfer, though? I'm not sure we can expect its ability to approximate physics in video form would transfer to any other mode (text, code, problem solving etc)

[ricopags]

depends on the hyperparams but one of the biggest benefits of a latent space is transfer between modalities