Hacker News new | ask | show | jobs
by wesm 2147 days ago
Arrow:

* Standardizes binary interop and "serialization" of large structured data, removing all conversions / serialization at ingest and export boundaries. This alone can mean > 2-100x performance improvement in an application that processes a lot of data

* The Arrow in-memory format is an ideal data structure to code analytical algorithms against.

Check out my 18min talk from a few years ago about the vision for the project https://www.youtube.com/watch?v=wdmf1msbtVs

1 comments

This is a great talk and should be a top-level comment explaining this release.