Hacker News new | ask | show | jobs
by fulmicoton 1145 days ago
This is indeed correct. Yes, we have comparatively few actors. One thing that is not explained in the blog post however, is that we can run several indexing pipeline on the same machine. We have a bunch of tricks in our framework to run ~1000ish index pipeline on the same server. That's a target of around 10k actors in the most extreme case.

We also send very few messages: the smallest message we send is represent batches of log lines.

As a result, our framework does not really care about performances. On one thread, it can process ~2.5 millions messages on my laptop: this is well above our needs, but it is correct to say that our framework is very slow compared to others.