|
|
|
|
|
by theLiminator
850 days ago
|
|
Hmm, how are you getting acceptable performance doing that? Is your data in a delta table or something? I've actually personally found that DuckDB is tremendously slow against the cloud, though perhaps I'm going through the wrong API? I'm using https://duckdb.org/docs/guides/import/s3_import. My data is hive partitioned, when I monitor my network throughput, I only get a few MB/s with DuckDB but can achieve 1-2GB/s through polars. Very possible it's a case of PEBKAC though. |
|