Hacker News new | ask | show | jobs
by tluyben2 1877 days ago
All analysis we run is over 100s of millions of objects (objects can be a few megabytes large) every time it runs. The slightest increase or decrease for 1 object obviously makes a huge difference overall; either in cost, time or both.
1 comments

Absolutely, but I don't feel the vast majority of corporations are doing that type of computation. I feel that even with hundreds of millions of rows Python can be a great solution (I have done multiple projects generating fairly complex projections from a few hundred million rows) for most projects.