[taeric]

Apologies for not getting back on this yesterday. Made the mistake of posting from my phone right before evening plans took over.

Looking at this, it seems you build a few indexes, such that I'm guessing those are the main drivers on the benefits? Makes sense, does it add to the space at all? As I said, most teams I work with are still on CSV, so even if this adds, I'm sure it is well below that.

At any rate, thanks for the response. Looks really nice!

[eddyxu]

No worries at all. For the teams which are happy on CSV/JSON, i'd admit that lance is not ideal alternative for them.

> It seems you build a few indexes, such that I'm guessing those are the main drivers on the benefits?

Yeah, we are building different indices into this columnar storage format, which is actually a happy side-effect of its good random-access performance. It does occur extra space for indices.

Thanks for your kind word too!

[taeric]

I'd hazard that many of the teams aren't so much happy with CSV, as they are ignorant of its costs. I fought for a bit to get them to move to parquet, but all too often they insist on having it in a format that excel can open.