Hacker News new | ask | show | jobs
by yoricksijsling 1262 days ago
Precisely. Conduits basically do everything on demand.

For parallel processing the backpressure becomes a bit more complicated, in particular if you end with a sequential consumer. If you're just doing your parallel tasks when a sequential consumer demands it, you'll find yourself still doing one task at a time. The straightforward solution is to work ahead a bit, spawning just enough parallel work to fill a small buffer that the consumer can read from. You get backpressure this way, but may have done a little bit too much if the consumer doesn't want the rest of the data. We'll show this in the third blog post of the series.

1 comments

> We'll show this in the third blog post of the series.

Thanks for the pointer! Will have a look.