[timhigins]

This has been researched quite a lot with the adjacent task of structure from motion (2d videos) - see NeRFs, Gaussian Splatting - and now there are a few models that do 3d from single 2d images fairly well.

[mhl47]

And interestingly it's still not even close to what a good human artist could do in this task having the same data.