Ask HN: How to scale long-lived connections with Kubernetes?

[k888sss]

My application serves users via long lived TCP connections and users are quite volatile. They come and go without any obvious patterns. And my current approach of scaling based on CPU/memory usage are too damn barbaric. I should not break active connections, but I also need to control the cost. On top my head, I can think of writing a custom middleware to manage connections and pods. Are there any ready-to-use solutions or approaches to tackle similar problems? Thanks

[wmf]

This kind of issue comes up in game development where I think the standard answer is a tier of proxies that can hold open the connection to the clients while reconnecting to different backends.

[k888sss]

Unfortunately, client code is not upgradable, and I have to solve this issue from server side only.

[neonlex]

I don’t understand your answer, that’s basically what wmf described. Put a proxy or load balancer in front of your service.

[k888sss]

Thanks. I can't upgrade clients to add reconnect functionality, and I can only work on the server side to solve the scaling issue. I think I should refactor my service as you mentioned by separating dedicated connection-managing servers and stateless worker servers such that I can scale workers.

[dominotw]

> I think I should refactor my service as you mentioned by separating dedicated connection-managing servers and stateless worker servers such that I can scale workers.

thats exactly what the original suggestion seemed to be was.

[k888sss]

Yupp. I think I misunderstood at first. Now I get it. Thanks all.

[thesandlord]

I know Agones[0] tackles this problem for game servers on Kubernetes. It could probably be used for this use case also, but it might also be overkill.

[0] https://github.com/GoogleCloudPlatform/agones

[k888sss]

Thanks a lot. This looks so promising. I am reading the docs :)