Hacker News new | ask | show | jobs
by thinkharderdev 1494 days ago
Arrow has union types (as well as structs and dictionary types). Parquet doesn't but I think it has an intentionally shallow types system to allow flexibility in encoding. Basically everything is either a numeric or binary and the logical type for binary columns is defined in metadata. So you can use, for instance, Arrow as the encoding.
1 comments

Yes, the comparison with Arrow ecosystem should really be more in depth since that's the closest thing that exists.