| > such like socket programming That's one of my biggest pet peeves (and if you see my other comments, you'll notice I have quite a few). To do socket programming in asyncio, you can either use: - protocols, with a nice reusable API and an interface that clearly tells you where to do what. But you can't use "await". You are back to creating futures and attaching callback like 10 years ago. - streams, where you can use async / await, you but get to write the entire life cycle by yourself all over again. I get that protocols are faster, and match Twisted model, and I get that streams are pure and functional, but none of this is easy. I use Python to make my life easier. If I wanted extreme perfs I'd use C. If I wanted extreme pureness I'd use Haskell. > wrapping non-async-compatible libraries and separating cpu-intensive blocking tasks to awaitable threads That's the one of the things asyncio did right. Executors are incredibly simple to use, robust and well integrated. Problem is: they are badly documented and the API is awkward. I won't write a tutorial in HN, but as a starting point: You can use: loop = asyncio.get_event_loop()
future = loop.run_in_executor(executor, callback, arg1, arg2, arg2...)
await future
If you pass "None" as an executor, it will get the default one, which will run your callback in a thread pool. Very useful for stuff like database calls.But if you want CPU intensive task, you need to create an instance of ProcessPoolExecutor, and pass it to run_in_executor(). I say it's one of the things asyncio did right because the pools not only distribute automatically the callbacks among the workers of the pool (which you can control the number), but you also get a future back which you can await transparently. |
That's my main problem with asyncio right now, bumping into problems and trying to find how to fix them by looking into the documentation is rather difficult. The documentation right now feels more like a documentation for an unsupported old library.
Also it's awkward that you cannot resolve kwargs on the functions that you pass to asyncio, like the callback in run_in_executor. You have to wrap the function in a partial resolving all kwargs and then send it to the executor.