Hacker News new | ask | show | jobs
by MrPowers 2220 days ago
Newer versions of Pandas don't even need fastparquet anymore. This code works:

import pandas as pd

df = pd.read_csv('data/us_presidents.csv')

df.to_parquet('tmp/us_presidents.parquet')

1 comments

Nice! Does that work alongside reading in via chunks and writing via row_groups? If I have a 500GB CSV will it work?