|
|
|
|
|
|
by HumanOstrich
801 days ago
|
|
|
Probably still need to stream or chunk the data instead of dealing with giant arrays to get decent performance. That's not a language/platform issue. I recently had to process and aggregate metrics for 5 million rows of user data (a few GBs) on my MacBook with Node.js. By streaming / iterating over the items without loading them all at once it chewed through them all in a few seconds. ¯\_(ツ)_/¯ And it's single-threaded (except for I/O offloaded to threads -- I'm talking about the calculations). |
|
|