[savant_penguin]

Being able to incrementally load compressed files. Sometimes I have to load large tables, do some processing and save the result. The easiest way to do that has been to load the rows of csv files by blocks (it's really easy to load parts of csv files). But I'd love a way to do the same row wise loading with compressed file formats such as parquet. (Bonus points if that's possible even if I'm not the one who built the original file)

[wcarss]

Not applicable to parquet, but I have used zless/zgrep often on tar.gz-compressed logs.

(I'm not sure what they're doing is actually that efficient though...)

[tstack]

FYI, the Logfile Navigator (https://lnav.org) can display gzipped/bzipped files on the fly. The top-of-tree has new support for automatically unpacking archives to a temporary directory using libarchive and displaying the contents.

[fwsgonzo]

https://linux.die.net/man/1/zless

Sadly, it looks like it just pipes to gzip -cdfq.