[jochalek]

Sounds like something that could be implemented with llm-d, though I've not experimented with it.

https://llm-d.ai/blog/intelligent-inference-scheduling-with-...

[rgthelen]

Yeah, I don't see why we could not integrate that. I think that is the next step as we move our workloads to production.

[mhamann]

`lf deploy` here we come!