|
|
|
|
|
|
by bourse_lee
38 days ago
|
|
|
What drives such a high throughput difference between Quack and Arrow on high-volume operations ? I'll try to search from source/Github, reply appreciated though, for example: - when DuckDb bulk exports a table, does Quack benefit from pre-existing compression/encodings/0-copy where Arrow requires decode+re-encode ? - the post mentions parallel reads, is the level of parallelism the same on Arrow vs Quack here ? Running the high throughput benchmark at resource saturation with increasing number of concurrent bulk-read clients would be more transparent |
|
|