[lmeyerov]

Yeah as soon as we write the word 'thread' or thinking about LLM API concurrency control across many user requests, all frameworks we tried are basically a wall instead of an accelerator. For a single user demo video on Twitter or a low-traffic streamlit POC to get a repo with lots of star gazers, they work quite well, and that's not far from what someone needs for an internal project with a small userbase. Just once this is supposed to be infra for production-grade software, the tools we have tried so far are still prioritizing features over being a foundation.