Hacker News new | ask | show | jobs
by kapsel 4287 days ago
I think this was a pretty dumb experiment, and the outcome was to be expected.

There will always be some sort of noise, pixels aligned differently etc., in a production like a tv series, and expecting the encoded output to be identical/matching on a blocklevel is pretty naive to say the least.

I did some experimenting using ZFS deduplication on MPEG2 files, where I encoded hundreds of MPEG2 dvd-sized videos, where 90% of the material was identical (the last 10% was affected by applying different watermarking techniques to the footage), and got some decent deduplication ratio (x1.2:1 or so).. But ZFS deduplication is expensive in memory/SSD, and it was definitely not worth it.

2 comments

It may be a "dumb experiment" if you already understand how the underlying systems (both the file system, and video encoding) work - but the author did not, and now having done the experiment, they understand more about them.
I guess you missed the part where I say almost the exact same thing as you just did, i.e. that I did never expect it to work?