[ibraheemdev]

> OS threads are expensive: an operating system thread typically reserves a megabyte of stack space

Why is reserving a megabyte of stack space "expensive"?

> and takes roughly a millisecond to create

I'm not sure where this number is from, it seems off by a few orders of magnitude. On Linux, thread creation is closer to 10 microseconds.

[eklitzke]

Yeah, none of this makes sense to me. Allocating memory for stack space is not expensive (and the default isn't even 1MB??) because you're just creating a VMA and probably faulting in one or two pages.

They also say:

>The system spends time managing threads that could be better spent doing useful work.

What do they think the async runtime in their language is doing? It's literally doing the same thing the kernel would be doing. There's nothing that intrinsically makes scheduling 10k couroutines in userspace more efficient than the kernel scheduling 10k threads. Context switches are really only expensive when the switch is happening between different processes, the overhead of a context switch on a CPU between two threads in the same process is very small (and they're not free when done in userspace anyway).

There are advantages to doing scheduling in the kernel and there are advantages to doing scheduling in userspace, but this article doesn't really touch on any of the actual pros and cons here, it just assumes that userspace scheduling is automatically more efficient.

[cmrdporcupine]

It's a cargo cult and a bias I see all over the place.

I feel like we're now, what, 20, 25 years on and people still haven't adjusted themselves to the fact that the machines we have now are multicore, have boatloads of cache, or how that cache is shared (or not) between cores.

Nor is there apparently a real understanding of the difference between VSS and RSS.

Nor of the fact that modern machines are really really fast if you can keep stuff in cache. And so you really should be focused on how you can make that happen.

[tcfhgj]

doesn't a async runtime have more knowledge about the tasks than the OS about the threads?

[ibraheemdev]

Not the runtime per se, but cooperative scheduling has the advantage that tasks do not yield at adverse code points, e.g., right before giving up a lock, or performing an I/O request. Of course the lack of preemption has it's own downsides, but with thread-per-request you tend to run into tail latency issues much earlier than context switching overhead.

[cmrdporcupine]

There's much ridiculous hatred for OS threads based on people's biases of operating systems and hardware from 20 years ago.

So much so that they'll sign themselves up for async frameworks that thread steal at will and bounce things all over cores causing cache line bouncing and associated memory stalls, not understanding what this is doing to their performance profile.

And endure complexity, etc. through awkward async call chains and function colouring.

Most people's applications would be totally fine just spawning OS threads and using them without fear and dropping into a futex when waiting on I/O; or using the kernel's own async completion frameworks. The OS scheduler is highly efficient, and it is very good at managing multiple cores and even being aware asymmetrical CPU hierarchies, etc.. Likely more efficient than half the async runtimes out there.

[tcfhgj]

hardware from 10 years ago - do you have benchmarks for more recent hardware?

https://vorner.github.io/async-bench.html

[cmrdporcupine]

I don't think those benches are much of a flex, even by the author's own description you'd be fine with any of them. They all have acceptable performance and don't show any order of magnitude differences or non-linear scaling problems.

Further, the benches that are showing best there are non-thread-stealing scenarios, not tokio.

I also suspect simply tuning the thread-based workloads more aggressively would have the same effect.

When I profile high throughput tokio applications there's way too much contention on shared atomics, mostly inside tokio's scheduler itself. On lower core count machines and where the workload is I/O heavy, this is probably fine. So, yes, web servers.

But I'm very interested in applications that scale on machines with lots of cores and where CPU is a large part of the equation.

[tcfhgj]

You assumed

> Likely more efficient than half the async runtimes out there.

The benchmark shows the opposite: 2 (multithreaded async runtime) vs 7 (threads) * 10^8 ns per request for 2k requests/s.

> non-linear scaling problems.

oh, look closely, the relative gap increases with #requests/s

[cmrdporcupine]

By TFA's own quote: "This means you probably shouldn’t put too much trust into the results I measured or base important decisions on the outcomes."

Your level of self-assured snark is way out of proportion to both the article and chart I'm looking at.

Props to the person who wrote the article. Your use of it however, is dubious.

[tcfhgj]

It's more appropriate than your snarks based on no data

[n_e]

> Why is reserving a megabyte of stack space "expensive"?

Because if you use one thread for each of your 10,000 idle sockets you will use 10GB to do nothing.

So you'll want to use a better architecture such as a thread pool.

And if you want your better architecture to be generic and ergonomic, you'll end up with async or green threads.

[lelanthran]

> Because if you use one thread for each of your 10,000 idle sockets you will use 10GB to do nothing.

1.On a system that is handling 10k concurrent requests, the 10GB of RAM is going to be a fraction of what is installed.

2. It's not 10GB of RAM anyway, it's 10GB of address space. It still only gets faulted into real RAM when it gets used.

[n_e]

> 1.On a system that is handling 10k concurrent requests, the 10GB of RAM is going to be a fraction of what is installed.

My example (and the c10k problem) is 10k concurrent connections, not 10k concurrent requests.

> 2. It's not 10GB of RAM anyway, it's 10GB of address space. It still only gets faulted into real RAM when it gets used.

Yes, and that's both memory and cpu usage that isn't needed when using a better concurrency model. That's why no high-performance server software use a huge amount of threads, and many use the reactor pattern.

[cmrdporcupine]

> Yes, and that's both memory and cpu usage that isn't needed

No, it literally is not. The "memory" is just entries in a page table in the kernel and MMU. It shouldn't worry you at all.

Nor is the CPU used by the kernel to manage those threads going to be necessarily less efficient than someone's handrolled async runtime. In fact given it gets more eyes... likely more.

The sole argument I can see is just avoiding a handful of syscalls and excessive crossing of the kernel<->userspace brain blood barrier too much.

[amluto]

> > Yes, and that's both memory and cpu usage that isn't needed No, it literally is not. The "memory" is just entries in a page table in the kernel and MMU. It shouldn't worry you at all.

Only if you never free one of those stacks. TLB flushes can be quite expensive.

[cmrdporcupine]

Fair enough, though it's not like an async tasks runner doesn't have its own often relatively expensive book-keeping.

[com2kid]

> 1.On a system that is handling 10k concurrent requests, the 10GB of RAM is going to be a fraction of what is installed

I've written massively concurrent systems where each connection only handled maybe a few kilobytes of data.

Async io is a massive win in those situations.

This describes many rest endpoints. Fetch a few rows from a DB, return some JSON.

[duped]

> you will use 10GB to do nothing.

You don't pay for stack space you don't use unless you disable overcommit. And if you disable overcommit on modern linux the machine will very quickly stop functioning.

[simonask]

The amount of stack you pay for on a thread is proportional to the maximum depth that the stack ever reached on the thread. Operating systems can grow the amount of real memory allocated to a thread, but never shrink it.

It’s a programming model that has some really risky drawbacks.

[matheusmoreira]

> Operating systems can grow the amount of real memory allocated to a thread, but never shrink it.

Operating systems can shrink the memory usage of a stack.

  madvise(page, size, MADV_DONTNEED);

Leaves the memory mapping intact but the kernel frees underlying resources. Subsequent accesses get either new zero pages or the original file's pages.

Linux also supports mremap, which is essentially a kernel version of realloc. Supports growing and shrinking memory mappings.

  stack = mremap(stack, old_size, old_size / 2, MREMAP_MAYMOVE, 0);

Whether existing systems make use of this is another matter entirely. My language uses mremap for growth and shrinkage of stacks. C programs can't do it because pointers to stack allocated objects may exist.

[dwattttt]

> C programs can't do it because pointers to stack allocated objects may exist.

They sure shouldn't exist to the unused region of the stack though; if they do, that's a bug (because anything could claim that memory now). You should be free and clear to release stack pages past your current stack pointer.

[matheusmoreira]

High level languages have entire runtime systems dedicated to managing resources like that. My language can allocate, grow, shrink and deallocate stacks dynamically. It has complete visibility into everything, and the stacks themselves are designed to be relocatable and position-independent.

In C it's impossible to even get the stack pointer without dropping to assembly or using compiler builtins. It's hard to know where the stack starts or even how big it is.

[simonask]

There isn’t any operating system or compiler that does this today, and it probably isn’t worth it to pursue. Enlarging the stack via page fault is really expensive, so you would need really advanced heuristics to prevent repeatedly unmapping/remapping those pages.

The correct tool for myriad of small tasks is coroutines / green threads / async tasks, so why spend any energy optimizing threads for that purpose instead of what they are already good at?

[simonask]

Stack memory is never unmapped until the thread terminates as far as I know. I don’t know of any kernel that does this, for precisely the reason you arrive at by the very last sentence.

[matheusmoreira]

It's just normal pages to the kernel. In theory, it's totally possible for the program to munmap some of its own stack's pages if it was sophisticated enough. Typical C programs just aren't capable of it, at least not without great effort.

[wmf]

On a 64-bit system, 10 GB of address space is nothing.

[matheusmoreira]

10 GB of RAM is certainly something though. Especially in current times.

[monocasa]

Except if those threads are actually faulting in all of that memory and making it resident, they'd be doing the same thing, just on the heap, for a classic async coroutine style application.

[asdfasgasdgasdg]

If you have hugepages enabled, all of those threads are probably faulting in a fair amount of memory.

[monocasa]

Only if you've actually faulted in 2MB contiguously already.

[matheusmoreira]

1 megabyte stacks mean ten thousand threads require 10 gigabytes of RAM just for the stacks. The entire point of the asynchronous programming paradigm is to reclaim all of those gigabytes by not allowing stacks to develop at all, by stealthily turning everything into a hidden form of cooperative multitasking instead.

[monocasa]

Only if they're resident. Otherwise you just need one page per thread of physical memory (so ~40MB on x86) and 10GB of virtual memory.

[matheusmoreira]

While that's strictly true, resident memory in this context is a function of worst case memory usage by the code executing on those stacks. Seems wise to assume worst case performance when discussing this.

The program could use one page's worth of stack space, which is optimal. The program could use like 200 bytes of stack space, which wastes the rest of the page. The program could recurse all the way to 9.9 MB of stack usage, stop just before overflow and then unwind back to constant 200 bytes stack space usage, and never touch all those pages ever again.

[jandrewrogers]

The author doesn't fully justify the assertion but it does have sound basis.

While virtual memory allocation does not require physical allocation, it immediately runs into the kinds of performance problems that huge pages are designed to solve. On modern systems, you can burn up most of your virtual address space via casual indifference to how it maps to physical memory and the TLB space it consumes. Spinning up thousands of stacks is kind of a pathological case here.

10µs is an eternity for high-performance software architectures. That is also around the same order of magnitude as disk access with modern NVMe. An enormous amount of effort goes into avoiding blocking on NVMe disk access with that latency for good reason. 10µs is not remotely below the noise floor in terms of performance.

[magicalhippo]

> Why is reserving a megabyte of stack space "expensive"?

Guess it's not a huge issue in these 64-bit days, but back in the 32-bit days it was a real limitation to how many threads you could spin up due to the limited address space.

Of course most applications which hit this would override the 1MB default.

[delusional]

> Why is reserving a megabyte of stack space "expensive"?

Equally, if a megabyte of stack is a lot for your usecase, can't you just ask pthreads to reserve less? I believe it goes down to like 16k

[joshsegall]

Author here. I definite brought some old bias and history to this. At the time address space was more limited (32-bit) and kernel interfaces were O(n), but you're right that these issues are not as relevant today and the numbers don't add up. I'll make a correction.