Hacker News new | ask | show | jobs
by cowboyd 1551 days ago
I think the simplest expression of the idea of structured concurrency that I've come across is this: The scope of any concurrent process is the same as the lexical scope of the function that created it. This has profound implications.

Let's say, for example that I call `doTasks()` with 10 separate tasks. Before any of them can complete, one of them fails. What happens to the other 9? In your example, they are "leaked" because they will continue running, even though the scope that was awaiting their results in gone. Using a "worker pool" as in the example, it might go something like this:

  async function doTasks(tasks = []) {
    let workerpool = new WorkerPool();
    try {
      for (let task of tasks) {
        workerpool.exec("my task", task);
      }
      let results = await workerpool.all();
      // do stuff with results
    } finally {
      // no matter the outcome, nothing outlives this scope.
      await workerpool.destroy();
    }
  }
It should be noted that cancellation, i.e. the ability to "destroy" a concurrent task is a necessary primitive for structured concurrency. You can think of it as the equivalent of automatically releasing memory at the end of a lexical scope.

There is already an implementation of Structured Concurrency for JavasScript here https://frontside.com/effection

1 comments

This helps complete the picture, thanks. In my current use-case I am batching up precomputing some expensive calculations in a genetic algorithm which all needs to be finished before I can run the competition. If one of the phenotypes throws an error Promises.all will catch it and my program will continue and I can handle the error, but as for my workers, yup they will leak - and I can't stop that using promises / async / await without leaking some internals of my workers with some shared semaphore or something, which is so 1990's. So yours this was a perfect example that helped bring it all together.

Funnily enough after this thread I did some digging around and came across effection as well, I'd never come across the term "structured concurrency" before but it feels like a natural next step on what I'm trying to accomplish, so I'll definitely be keen to see more development in this area and maybe some first class language constructs at some point.

Thanks for taking the time to give an example and explain the concept in relatable terms.