[xiaodai]

For larger-than-RAM data I would recommend diskframe.com

It uses dplyr and data.table syntax to manipulate data on disk

[phillc73]

I've not used diskframe.com, but from experience can recommend the 'fst'[1] file format with 'fsttable'[2] for reading on disk data tables.

[1] https://github.com/fstpackage/fst

[2] https://github.com/fstpackage/fsttable

[xiaodai]

disk.frame uses fst as the underlying format

[meztez]

Thanks, so far we just scaled up our vm ram but i might find a use for it.