[belthesar]

Something of note though is that things are shot for the aspect ratio they're shooting for. I'm guessing The Shield was intentionally shot with both Widescreen and 4:3 in mind if that happened, because it's pretty common to look through your viewfinder as a cinematographer, see that there's a boom in the shot, but it's outside of the matte and proceed with filming.

If we were to trust AI to do this, we'd need to build models specifically for this, with a way to give context to the rest of the set for the scene in question. From a technical perspective, I find the concept interesting, but from a practical sense, the amount of energy and time required to do this is almost a non-starter, and in doing this, we'd be going beyond the director's intent for the work at hand, which makes a transformative interpretation a degree or two removed from the intent of the director and production team.