After watching the video I was trying to find out just how much one of those microscopes cost. Couldn’t find a price anywhere so I’m assuming it’s far out of my budget. But this kind of video is probably the greatest kind of ad there is, just genuinely showing how cool something is. Don’t have a use for it either though, but I would love to have one anyways.
This would line up with the approach talked about in the video. In very short terms, pictures were taken at various focus distances and height was defined as whichever distance was the sharpest. This would essentially make a 2D height map with all of the artifacts that would come with it
It's has 5x exaggerated height by default, so maybe that's what makes it look wonky. Looks way better by 1x.
If you really want to capture the full visual information of a painting, you'd need full PBR-style data — reflective, refractive, subsurface properties — essentially the response of the surface from any viewing angle in the hemisphere, lit from at least a few fixed directions (like in a museum light setup). Even limited to the visible spectrum, this would massively increase the amount of data needed to represent the image accurately.
The 2019 scan apparently deliberately removed reflections, even though they're an essential part of the artist's intended expression.
Are there models that simulate the actual physical properties of paintings — under artificial lighting, viewed from arbitrary angles? Seems like a worthwhile direction for preserving artworks beyond their flat 2D captures. It could also enable virtually accurate displays of art for single observers using head-tracked screens or VR.
Might also be a promising use case for NERFs or 3D Gaussian Splatting.