[mananaysiempre]

That is the ecological niche of Rayon (cited) as well, isn’t it? You need to process a lot of things (thousands to millions), you want to parallelize that processing as much as possible (couple dozen of cores tops), you want to not get killed by scheduling overhead. So you account for the per-thing overhead.

[CyberDildonics]

Anything can have its overhead amortized, but claiming that amortization as your actual overhead is just a lie.

[hansvm]

Most engineers aren't precise with throughput vs latency. Ideally you should report both figures (and anything else salient in performance-sensitive spaces), but it's less a lie and more an extremely commonplace mode of thinking and speaking.

Moreover, I think that mode of thought comes from the fact that most programming problems don't have hard latency bounds, so throughput dominates the conversation. If I'm spending 10us on average while handling a 10ms soft deadline, every single component can easily be occasionally 100x more expensive (latency) without me caring, and if it buys me another 1us on average (throughput) then I'll save gobs of money in compute.

[Dylan16807]

If you're trying to split a sub-second task across multiple threads, then latency is probably your main concern.

[hansvm]

Kind of, but not in a way that matters for this scheduler (at least, I posit, not usually). If you have a single task with a wall-clock time in the 10-1000ms range and want to make it "faster," yes, you probably want to improve the latency. However, the latency introduced by this experimental library is negligible on the timescales being considered. With that in mind, a throughput improvement is a better description of the sorts of engineering efforts you need to accomplish your goals.

From a slightly different perspective, one thing the library does on top of handling small workloads well is handling _finely grained_ workloads. That's super important from a usability point of view, since you can express the parallelism in the most natural way and let the library handle the fact that it's hard to schedule that task (e.g., most schedulers suck as soon as you're talking about work items on the order of a single cache line). Just being able to have a convenient abstraction when writing a long-running job is also a fantastic feature. In that case, we would still care about throughput, not latency.

Separately, though definitely more rarely, there are jobs which are sub-second but where it's hard to batch many of them at once. If you're forced to execute many and care about the time till completion (latency, again at a much longer timescale), being able to make each one much faster is a big deal.

I really think that first paragraph is a commonplace occurrence though. Something important is slow (measured in "human" blinking timescales), so you slap a better scheduler and some parallelism at it and start work on the next ticket. Ripgrep, at some level (it has lots of other technical accomplishments; I don't want to let this comment give the mistaken impression that I'm demeaning the project) is useful precisely because it throws parallelism at sub-second problems.

[CyberDildonics]

Most engineers aren't precise with throughput vs latency.

Anyone making claims about performance should know the difference.

This isn't even about either, it's lying about overhead by not counting it correctly.

If someone asks what your cable bill is and you say it's only $2.50 a month because you have 32 TVs, no one is going to say that makes sense.

[hansvm]

It's not "lying" to use the words most likely to correctly get your point across to your target audience. Maybe they could have communicated better (for a seemingly dead project, IMO they put enough time in regardless), but it's not lying.

> Anyone making claims about performance should know the difference.

They probably do know the difference. Knowing the difference isn't the thing you're quibbling with.

> If someone asks what your cable bill is and you say it's only $2.50 a month because you have 32 TVs, no one is going to say that makes sense.

Sure...because when asking about your cable bill it's unambiguous that you want the total number of dollars. The whole reason there's any issue at all here is that some of their language is ambiguous if you don't consider the target audience and don't analyze their charts or read the descriptions of those charts. Picking an analogy which resonates precisely because of the lack of ambiguity doesn't say a whole lot about the actual problem at hand.

[CyberDildonics]

The whole point of overhead is that it is a base constant on top of whatever you do. The whole point of overhead is to eventually be amortized in some way. When someone asks what the overhead is, it is a lie to try to factor amortization in because it's implied that it will be done somewhere anyway.

[hansvm]

I think we're maybe talking past each other with definitions.

I think something is a lie when somebody intentionally tries to mislead (even when communicating "facts" if the intent is to maliciously shape the recipient mind). I think when they intentionally try to convey the "right" ideas (as I think this author is doing -- again, using words the way you think most of your peers use them is usually a good thing), that's not a lie. There are gray areas in between. I don't think this is a gray area, except insofar as I don't know all the facts perfectly.

You still seem to disagree with me though. I'd really like to understand why. I have a couple theories, which I'll list, but I'm interested in any insights you have:

1. You disagree with my description of what "lying" is.

2. You think the author is less deserving of trust than I do (since they seem to be profiting almost nothing other than a tiny bit of publicity, since they did outline in their README exactly what they meant, with photos, and since the title didn't stoop to clickbait, my default position is that they likely weren't too malicious, but you might have different priors and might also disagree with my opinions of those conflicts of interest).

> the whole point of overhead ...

That's perhaps the last pertinent thing we disagree about. I'll throw out my opinion, and I sincerely want to know your thoughts on the matter:

An extremely commonplace occurrence is that something is slow and you want to speed it up (measuring perceived latency). If you can magically do that by adding parallelism then it's a good day at work. Unfortunately, parallelism can easily add "overhead" -- parallelizing small or finely grained problems usually increases wall-clock timing, usually substantially.

In that mode of operation, the whole problem you're trying to solve is reducing a certain latency measurement, making it clear from context what "overhead" means. Other definitions don't even enter into the equation.

Zooming out a bit, I've heard that phrasing a ton of times in many different companies. Sure, it's better if people know to speak more precisely (and I think we're in agreement that a few more performance characteristics would be nice to know here), but it's hard for me to imagine a world were my default perception of somebody is that they're lying when they're using the exact same words and meanings as scores of other good engineers. _Maybe_ they are, but that's by no means my default assumption.