Hacker News new | ask | show | jobs
by sshillo 3738 days ago
Are there any plans to support larger than RAM datasets? Like hdf5 or bcolz does.
1 comments

The format already supports larger than RAM data, but we don't yet have an API for creating those files or just extracting slices. That will come in the future.
Thanks for working on this. Really a great effort. Hope your guys can get good inspiration from projects like h5df, bcolz, pytables having the option of incrementally adding features and maintaining an open spec.