Hacker News new | ask | show | jobs
by FridgeSeal 13 days ago
> Increasing threads beyond system's hardware cores/threads resulted only in marginal gains of a couple of milliseconds worth of differences on huge workload with large increase in memory

Careful, if you say that too loudly, the "get rid of async just spawn more threads!!!!!" people will come out of the woodwork to yell at you about how _all_ async is a lie and we should instead pretend none of it exists and just spawn more threads.
1 comments
elendilm 13 days ago
I am now trembling at the thought of warriors who will skin me alive :).

Jokes aside, there are use cases for rayon, use cases for tokio async, use cases for "may" coroutines, use cases for a custom scheduling policy, or use cases for a combination of these.

We went with "may" coroutines (with its thread pinning) + custom numa work pinning (to a may thread) due to "may's" lightweight nature and not having to have our functions colorized. We use rayon where cache locality penalty is minor compared to the millisecond latency gains due to work stealing - but this is only for an edge case.

Our log broker Monolog (akin to kafka) which sits atop Dip uses tokio async because it plays nicely with zmq while "may" doesn't.

So I am a big fan of using the right tools for the job. Each technique has its own pros and cons. Informed decisions based on use cases matter above any dogma.

link
FridgeSeal 13 days ago
It sounds like you’ve got a super interesting stack going on there, evidently with a large performance/latency focus.

May (haha) I ask what this is in service of? I’m somewhat a fan of the thread-per-core model, so I’m curious as to what you’re doing with it.

link
elendilm 12 days ago
Our stack consists of two consumer facing mobile apps, Slyp and SlypBusiness, which necessitated developing the underlying infrastructure over time for scaling and efficiency.

That stack includes Dip (our database substrate), Monolog (our log broker), Singularity (our KV engine), Craft (a mobile app for app development using flow charts, snippets, and a WYSIWYG designer), and Portal (a mobile app for remote container management and a HITL AI workflow).

Craft and Portal are MVP-style functional, but aren't in production yet. Everything else is being rolled out gradually.

You're more than welcome. Happy to discuss any of it in more detail. Rather than digress further in this thread, feel free to reach out privately if you'd like to continue the conversation.

link