Hacker News new | ask | show | jobs
by cyanf 359 days ago
It's either sticky sessions or an lb that keeps track of prior sequences and route to the instance with the largest match. https://docs.sglang.ai/router/router.html