| This is a thoroughly interesting topic. Thanks for the article. I haven't thought much about iterators link to coroutines. As a hobby, I am working to write about a dream programming language. I happen to be really interested in parallelism, asynchronous, coroutines, multithreading and concurrency. I want: * seamlessly switch between remote-thread coroutine, local thread coroutine. * concurrency and parallelism and async to be easy to think about, reason about, read and program * programs should be easy to parallelise and be async and concurrent Go iterators seem to be local to a thread, but what if you want to distribute work across threads? I've been thinking of scheduling recently. Imagine you're a search engine company and you want to index links between URLs. How would you solve this with coroutines? task download-url
for url in urls:
download(url)
task extract-links
parsed = parse(document)
return parsed
task fetch-links
for link in document.query("a")
return link
task save-data
db.save(url, link)
How would you do control flow and scheduling and parallelism and async efficiently with this code?* `db.save()`, `download()` are IO intensive whereas `document.query("a")` and `parse` is CPU intensive. * I want to handle plurality or multiple items trivially such as multiple URLs and multiple links. * I want to keep IO and CPU in flight at all times. I think I want this schedule: https://user-images.githubusercontent.com/1983701/254083968-... I have a toy 1:M:L 1 scheduler thread:M kernel threads:N lightweight threads lightweight scheduler in C, Rust and Java https://github.com/samsquire/preemptible-thread This lets me switch between tasks and preempt them from user space without assistance at descheduling time. I have a simplistic async/await state machine thread pool in Java. My scheduling algorithm is very simple. I want things like backpressure, circuit breakers, rate limiting, load shedding, rate adjustment, queuing. |
