Hacker News new | ask | show | jobs
by karbarcca 2082 days ago
It's always a transcendental experience to move data from CSV into any more structured/flexible format!
1 comments
klhugo 2082 days ago
And fast! I got easily 3 orders of magnitude speedup. I used to load CSVs with pandas and do typical filtering operations... oh boy, some of the stuff would take a hole morning, not to mention the memory usage. I do all that in minutes now.
link
benhurmarcel 2081 days ago
Sorry if that sounds stupid, but then when you get a csv file you write it to an SQLite file before doing computations?
link