[vorticalbox]

Why not load the data into sqlite dB and let the clients query that? Is there a reason you're loading 10s/100s gb into memory?