Hacker News new | ask | show | jobs
by sradman 1934 days ago
Why this hybrid dataframe? [1]:

> [Arrow] adoption will take time, and most people are probably more comfortable seeing NumPy arrays. Therefore a Vaex version 4 a DataFrame can hold both NumPy arrays and Apache Arrow arrays to make the transition period easier.

There seems to be agreement that Apache Arrow is the future of dataframes across ML ecosystems. I didn't realize this transition impacted NumPy arrays in addition to Pandas dataframes in Python.

[1] https://vaex.io/blog/a-hybrid-apache-arrow-numpy-dataframe-w...

1 comments

> There seems to be agreement that Apache Arrow is the future of dataframes across ML ecosystems

Not, until there is proper multidimensional array support.