|
|
|
|
|
|
by sdz
1251 days ago
|
|
|
You're probably looking for the Arrow IPC format [1], which writes the data in close to the same format as the memory layout. On some platforms, reading this back is just an mmap and can be done with zero copying. Parquet, on the other hand, is a somewhat more complex format and there will be some amount of encoding and decoding on read/write. Flight is an RPC framework that essentially sends Arrow data around in IPC format. [1] https://arrow.apache.org/docs/python/ipc.html |
|
|