[weberc2]

> No amount of async/await or goroutines can solve this problem

Presumably he’s not serving the model but running it, which is cpu bound, in which case Goroutines would solve the problem.

[fnord123]

async would also solve his problem in Python. So there's Sanic, Quart, etc.

[weberc2]

Async doesn’t help with CPU-bound tasks, but yes, you can rig up systems that involve running multiple Python processes behind a load balancer. It’s just more work.