Hacker News new | ask | show | jobs
by pizza 1222 days ago
Coincidentally, the current winner of the Hutter Prize, Starlit, works by trying to come up with a traveling salesman path between wikipedia articles based on document embeddings + a context adaptive compressor.

To make a restartable codec using a non-restartable compressor just means you have to implement that layer not using the non-restartable compressor. As a side note, if I’m not mistaken, the zstd docs suggest that this is already possible - you just have to used the (advanced) buffer-less streaming api, I believe: http://facebook.github.io/zstd/zstd_manual.html#Chapter19