[vovavili]

This makes more sense. I still wonder if the author isn't just effectively recreating Apache Iceberg manually here.

[tamnd]

I intentionally kept it lightweight. Just Parquet files + simple partitioning + commits on Hugging Face. That already covers most of what I need, without introducing a heavier stack or extra dependencies.

Also, I wanted something that is easy to consume anywhere. With this setup, you can point DuckDB or Polars directly at the data and start querying, no catalog or special tooling required.

[tomrod]

Are they paying for the repo space, I wonder?

[cyanydeez]

someones paying to keep name dropping Iceberg(tm)

[mulmen]

Weird accusation. Iceberg is an Apache project. I don’t think anyone gets paid when you use it so not sure what the benefit of shilling would be. It is just a table format that’s well suited for this purpose. I would expect any professional to make a similar recommendation.