[amiantos]

https://aihorde.net is something like an open-source load balancer with queuing for LLMs, too