[noident]

> It got me wondering if it was actually possible to make Python’s async event loop work with multiple threads.

There is built-in support for this. Take a look at loop.run_in_executor. You can await something scheduled in a separate Thread/ProcessPoolExecutor.

Granted, this is different than making the async library end-to-end multi-threaded as you seem to be trying to do, but it does seem worth mentioning in this context. You _can_ have async and multiple threads at the same time!

[nbsande]

run_in_executor is pretty powerful for running sync code in async code, but my use case was more making async code utlize the cpu better. I think, just using run_in_executor would add a lot of complication and changes to how you use async await. But great point none the less!

[zo1]

I've had plenty of success using run_in_executor in production. You basically just use the decorator, and it mostly just works.

[sandeep1998]

Which decorator?

[zo1]

I went back to look at some of the old code as my memory was hazy. It was tornado's implementation of the run_on_executor method that we used, which is used as a decorator.

https://www.tornadoweb.org/en/stable/concurrent.html#tornado...

[noident]

> but my use case was more making async code utlize the cpu better

But run_in_executor achieves that as well! If you use no-GIL Python and a thread pool (or GIL Python with a Process pool), you will utilize more CPU cores.

[d0mine]

run_in_executor that can be spelled as asyncio.to_thread() won't help utilizing cpu for pure Python code. It may help with blocking I/O, or cpu-heavy C extensions that release GIL. Otherwise, you have to utilize different processes to take full advantage of cpus (there are also subinterpreters (same process, separate GILs) but that exotic).

[noident]

>run_in_executor that can be spelled as asyncio.to_thread() won't help utilizing cpu for pure Python code

That isn't true. If you use a ProcessPoolExecutor as the target instead of the default executor, you will use multiple processes in pure Python code.

[d0mine]

the default executor must be ThreadPoolExecutor. But you are right you can pass ProcessPoolExecutor too.

https://docs.python.org/3/library/asyncio-eventloop.html#asy...