Hacker News new | ask | show | jobs
by tn13 4276 days ago
I am not sure if size of data and clusters is something people could brag about as if it is "bigger the better".

I am more interested in seeing what kind of engineering efforts they are putting to minimize the size of data and clusters.

1 comments

Soooo true. In my former life I frequently came across traders or quantitative analysts who wanted to use "big data stuff" to analyze their stupid ideas. I never gave these people clusters of Hapoops or whatever they'd been reading about on the blogs. Usually what I did is reformatted their data files into column-oriented systems (like Vertica) or varint+delta-encoded timeseries that would fit into a single CPU's cache memory.