Hacker News new | ask | show | jobs
by theropost 411 days ago
I need this, just finished 300GB of CSV extracts, and manipulating, data integrity checks, and so on take longer than they should.
1 comments
constantcrying 411 days ago
Why wouldn't you use a data format meant to store floating point numbers?

HDF5 gives you a great way to store such data.

link
theropost 411 days ago
Sounds interesting, I'll give it a look. I'm unfortunately limited to CSV, XML, or XLS from the source system, then am transforming it and loading it into another DB.
link