|
|
|
|
|
|
by sammysidhu
762 days ago
|
|
|
We're actually using pyiceberg to retrieve metadata! All our IO and decoding happens in the rust side once the data has been passthrough. We expose something called a ScanOperator which allows integration into various catalogs through a thin layer that exposes ScanTasks. Iceberg's impl: https://github.com/Eventual-Inc/Daft/blob/416009138359a9d410... |
|
|