Hacker News new | ask | show | jobs
by drej 1687 days ago
I remember benchmarking fast compression algorithms a while back and had lz4 as one of the top contenders, but I remember going with Snappy over lz4, because I ran into some compatibility issues in lz4 land... I can't recall any specifics, but I think it was some Hadoop libraries not being able to read files lz4-compressed by some other versions of the library.

Has anyone run into the same issue? I'm considering reopening this investigation (even though I'm very happy with Snappy).

1 comments

Similar problem I've encountered is AWS Athena not supporting Parquet files compressed with Zstd. I re-wrote files with Snappy to get around it.