[argonaut]

These results: http://richzhang.github.io/colorization/ from Berkeley are much better than this model (and the code is open source as well).

Furthermore, those better results have the advantage of a much simpler model. This model has a fairly complicated architecture (a complex residual concatenation setup) and many more parameters (I would guess anywhere between 2x-10x as many, but I'd have to take a closer look), which means it's much slower to run and takes up more memory (disk and RAM).

I'd also say that in general the better model does things that are a lot more common sense: using the CIE Lab color space (perceptually uniform), omitting pooling, using a classification loss instead of regression (regression in generally performs poorly in deep learning), etc.

[logicrook]

And the thread concerning that much better model: https://news.ycombinator.com/item?id=11403653

[im3w1l]

The authors of that point out that those are the especially successful cases. Presumably the the new article also showcases the success.

For comparability, I think it would be best if we could see outputs for the two models for the same, chosen in advance and not cherry picked, images.

[specialist]

Idle thought: Could colorization be used to (lossy) compress full color images (usefully)?

What I imagine is full color input -> create B/W & color histogram (list of colors used) -> image viewer uses colorization algorithm to reapply colors.

[adrusi]

I don't really know a ton about image compression, but I exported the same photo to jpg using the same settings twice: once with 3 channels and once desaturated and then with two channels hidden (I couldn't figure out how to make a 1-channel jpg in gimp, no sure if it's possible). The 1-channel export had less than 30% file size reduction.

I don't think a compression technique that would require that much processing power and have that little size reduction would be too useful.

[maffydub]

A 30% saving is roughly what I would expect.

Commonly JPEG separates the image into 1 channel of luma and 2 channels of chroma. It then downsamples the chroma to half the luma resolution, meaning that you have twice of much raw luma data as you have chroma.

It then goes on to do a whole load of fun discrete cosine transform, quantization and huffman encoding, but to a first approximation, I'd expect them to compress roughly similarly on average.

Since you've got twice as much luma data as chroma data, dropping the chroma will only save you ~30%.

Since the predictions from the neural network are not always accurate, you could then encode the error in the compressed file, and restore it on decompression. This will generally be close to 0, so compression should be pretty good.

At the end of the day, 30% better compression would be great... if the processing overheads aren't too great - I think that's the key factor.

(Most of this is from memory based on some hacking I did to be able to transcode videos to play on a cheap Chinese iPod Nano clone, which used an undocumented variant of MJPEG. The default quantization tables for luma and chroma are different from each other. The iPod Nano clone was using the standard quantization tables but the other way round (so using the luma table for chroma and the chroma table for luma). I can only imagine this was a bug in their code, as it was bound to reduce their compression ratio/image fidelity.)

[Lerc]

I think you could do quite well with seeding areas with known colours. Provide data of x,y,colour at a few sample points where the initial guess was too far out. A very small amount of data might allow for quite accurate results.