[treblig]

"Postproduction was mostly refining creative choices that we were not able to finalize on the set in a way that we deemed photo-real."

Does anyone know how they were able to swap out the in-camera version of the background originally shown on the LED wall with something more convincing later? Seems like it'd be tough since it's not a green screen!

[janekm]

While currently they are using "green screen" in those instances, given that the camera positions are already being tracked, and the image displayed on the screens is known, it would be possible to re-render what image the camera should have seen if the foreground elements hadn't been present and use the difference with the recorded image as a mask for further post-processing.

(which would be very cool as it would also allow using a low-resolution version of the background during production that could then be re-rendered with a higher resolution image after the fact)

[estebank]

It seems to me that making the system be able to perform masking itself (instead of projecting a green screen) would reduce the ability to completely replace the scene easily, but the reflections already give you that problem. The advantage you would have is that the real elements in the scene would no longer need any feathering around the edges causing them to look hazy, because now you can take a few pixels of the projected background as a transition to the higher resolution render.

[czr]

iirc they project small green region around the actors and real props, so that ambient light and reflections are still mostly correct but they can also get clean matte out.

[re]

There's a demo of this here: https://youtu.be/Hjb-AqMD-a4?t=609