[adhocpotato]

I'll add some more detail. There are a lot of different ways your application can break as it scales up. From something which is handles data flow, the three I tend to think about the most are: the model, throughput and number of connected clients.

We've tested the model with 100k identities with the operations that clients use: auth, listing services, creating sessions, etc, to make sure that the model scales reasonable well. We had to add additional denormalization and make some other tweaks, but now the controller scales relatively well for those operations. I'm sure there are still edge cases where it may break down, but we'll have to fix them if someone hits them.

We've done thoughput testing to make sure we can handle high throughput use cases. This also resulted in lots of changes, including reworking the end-to-end flow control. This is an area where we're happy with the progress we've made, and performance is in a reasonably good place, but where we have lots of ideas of how to continue to improve and will be continuing to test and iterate.

Having tons of connected clients (even without much traffic flowing) is it's own scaling challenge. We've done some amount of testing here. As part of the testing mentioned above we had to make some changes to make sure that slow clients didn't hold up fast clients. More generally,this is where things start getting complicated and very specific. You can have very different types of traffic flow, so it's hard to model anything generic. We have not done as much in this area because we've not seen any cases where we're memory constrained, which is the usual sign of a concurrent connections scaling bottleneck.

Hope that's helpful.