Hacker News new | ask | show | jobs
by AymanB 751 days ago
I was wondering, any way to use it with distributed systems for data analytics?

Imagine a set of workers that ingest data in parallel, would that work?

Currently it's pretty simple and i am spawning a process within the worker so it reads some stuff such as memory usage, cpu usage etc... But I would like to improve it.

1 comments

Not entirely sure what you're asking about exactly, but, well, currently the analysis part can only run on a single machine, and it was never designed to run on multiple machines. (Which, for analyzing bigger dumps can be a problem if you don't have a lot of RAM.)

It could probably be done, but the analyzer would have to be mostly rewritten. (Which I currently have no plans to do.)