Hacker News new | ask | show | jobs
by bobbylarrybobby 1968 days ago
Is this merely planned, or does pandas now use Arrow’s format? I was under the impression that pandas was mostly numpy under the hood with some tweaks to handle some of the newer functionality like nullable arrays. But you’re saying that arrow data can be used by pandas without conversion or copying into new memory?
1 comments

Pandas is still numpy under the hood, but you can create a numpy array that points to memory that was allocated elsewhere, so conversion to pandas can be done without copies in nice cases where the data model is the same (simple data type, no nulls, etc.): https://arrow.apache.org/docs/python/pandas.html#zero-copy-s...