[lukehollis]

Something close to what you mention is dynamic scene capture, such as this paper: https://dynamic3dgaussians.github.io/

You can manipulate the splats in viewer, kind of similar to a pointcloud, something to the effect of https://twitter.com/parazar/status/1713618668788109357 but there are a bunch around social media.

This has an entrypoint to the research and various projects so far: https://github.com/MrNeRF/awesome-3D-gaussian-splatting