Hacker News new | ask | show | jobs
by brudgers 99 days ago
“Your data isn’t big” is a good working definition of big data.

Google has big data. You are not google.

1 comments

I think the definition of big is smaller than that. Mine was "too big to fit on a maxed-out laptop", effectively >8TB. Our photo collection is bigger than that, it's not 'big data'.

Or one could define it as too big to fit on a single SSD/HDD, maybe >30TB. Still within the reach of a hobbyist, but too large to process in memory and needs special tools to work with. It doesn't have to be petabyte scale to need 'big data' tooling.

“Your data is not big” comes from this thread…https://news.ycombinator.com/item?id=7192839

8TB is a couple hundred hours of 4k RAW video assets.

This is true, but 8TB is big data if it's text.