[boomlinde]

One thing I would consider "unclean" about the zio approach (and e.g. libtask) is that you pass it an arbitrary expected stack size (or, as in the example, assume the default) and practically just kind of hope it's big enough not to blow up and small enough to be able to spawn as many tasks as you need. Meanwhile, how much stack actually ends up being needed by the function is a platform specific implementation detail and hard to know.

This is a gotcha of using stack allocation in general, but exacerbated in this case by the fact that you have an incentive to keep the stacks as small as possible when you want many concurrent tasks. So you either end up solving the puzzle of how big exactly the stack needs to be, you undershoot and overflow with possibly disastrous effects (especially if your stack happens to overflow into memory that doesn't cause an access violation) or you overshoot and waste memory. Better yet, you may have calculated and optimized your stack size for your platform and then the code ends up doing UB on a different platform with fewer registers, bigger `c_long`s or different alignment constraints.

If something like https://github.com/ziglang/zig/issues/157 actually gets implemented I will be happier about this approach.

[aidenn0]

Maybe I've been on x64 Linux too long, but I would just specify 8MB of stack for each fiber and let overcommit handle the rest. For small fibers that would be 4k per fiber of RSS so a million fibers is 4GB of RAM which seems fine to me?

[Hendrikto]

Couldn’t you use the Go approach of starting with a tiny stack that is big enough for 90% of cases, then grow it as needed?

[lukaslalinsky]

Go depends on the fact that it can track all pointers, and when it needs to resize stacks, it can update them.

Previous versions of Go used segmented stacks, which are theoretically possible, if Zig really wanted (would need compiler support), but they have nasty performance side-effects, see https://www.youtube.com/watch?v=-K11rY57K7k

[loeg]

Resizing stacks on use does not depend on any of these properties of Go. You can do it like this in C, too. It does not require segmentation.

[boomlinde]

Resizing stacks insofar that expansion may require moving the stack to some other place in memory that can support the new size depends on these properties. Your initial 4k of coroutine stack may have been allocated some place that wont fit the new 8k of coroutine stack.

Or are you making a point about virtual memory? If so, that assumption seems highly platform dependent.

[loeg]

You would implement this with virtual memory. Obviously, this is less of a limited resource on 64-bit systems. And I wouldn't recommend the Go/stack/libtask style model for high concurrency on any platform.

[lukaslalinsky]

I'm very interested to know how. Do you mean reserving a huge chunk of virtual memory and slowly allocating it? That works to some degree, but limits how many coroutines can you really spawn.

[loeg]

Yes, exactly.

[boomlinde]

Consider that resizing the stack may require reallocating it elsewhere in memory. This would invalidate any internal pointers to the stack.

AFAIK Go solves this by keeping track of these pointer locations and adjusting them when reallocating the stack. Aside from the run-time cost this incurs, this is unsuitable for Zig because it can't stricly know whether values represent pointers.

Go technically also has this problem as well, if you for example convert a pointer to a uintptr, but maintains no guarantee that a former pointer will still be valid when converted back. Such conversions are also rarely warranted and are made explicit using the `unsafe` package.

Zig is more like C in that it gives the programmer rather than a memory management runtime exclusive control and free rein over the memory. If there are some bits in memory that happen to have the same size as a pointer, Zig sees no reason to stop you from interpreting them as such. This is very powerful, but precludes abstractions like Go's run-time stack reallocation.

[loeg]

8kB is enough for 90% of use cases. But then you invoke getaddrinfo() once and now your stack is 128kB+.