[neuromantik8086]

> And yes, I'm definitely interested in doing video

As someone familiar with the libraries space, I'd actually be very interested in seeing a machine learning model that could deal with "cleaning up" old film (I've actually brought this up w/ several of my ML friends occasionally). One of the biggest challenges in the world of media preservation is migrating analogue content to digital media before physical deterioration kicks in. Oftentimes, libraries aren't able to migrate content quickly enough, and you end up with frames that have been partially eaten away by mold.

As a heads-up, these are some of the problems you might encounter on the film front (which you might not otherwise find with photos due to differences in materials used, etc):

https://www.nyu.edu/tisch/preservation/program/05fall/physic...

https://www.filmpreservation.org/preservation-basics/vinegar...

[nkoren]

I believe that Peter Jackson's recent endeavour in cleaning up WW1 footage employs significant ML for de-noising, frame interpolation, and colorising. I haven't seen the final film, but some of the clips are staggeringly good: https://www.bbc.com/news/av/entertainment-arts-45884501/pete...

Edit: Here's maybe a better link -- https://www.bbc.com/news/av/entertainment-arts-45803977/pete...

[lcrs]

I'm actually not sure much ML was involved here - depends where you draw the line I guess, but denoising and interpolation for restoration typically use more traditional wavelet and optical flow algorithms. The work for this was done by Park Road Post and StereoD, which are established post-production facilities using fairly off-the-shelf image processing software. The colorisation likely leant heavily on manual rotoscoping, in the same way that post-conversion to stereo 3D does.

I'd love to hear otherwise but I'm not aware of any commercial "machine learning" for post-production aside from the Nvidia Optix denoiser and one early beta of an image segmentation plugin.

[nkoren]

Huh, I recall seeing an article at one point (can't find the link) where it said or suggested that ML was involved. Of course this could have just been a journalist failing to make the distinction; I've seen everything from linear regression on up naively lumped into the ML bucket.

In any case the results are damned impressive -- can't say I've seen anything like it before.

[WalterBright]

Thanks for posting this, it's incredible work!