[xiaodai]

I mean, don't reach for Spark. Just get Dask, a databased or R's disk.frame is mostly fine.