| These seem to have been "technical" bugs, mostly in the client-server communication of the self-service websites. But hosting management also has a whole lot of potential for logical errors. I work at a B2B ISP that is also a managed service provider, and every now and then somebody calls for more self-service. I'm not against it, but it's full of landmines. It starts with such simple things as account creation. We allow accounts with the appropriate roles to be used for several different customers, which means we cannot namespace them by tenant. We used to give accounts usernames based on the first and last name, but that makes it very easy to leak which other accounts exists (when "John Doe" becomes "johndoe4" instead of just "johndoe", you can infer the existence of other accounts, and thus that other "John Doe"s are among our customer base). We had to change that, which isn't as easy as it sounds for a company that has been doing things a certain way for 25 years. Potential logic errors and information leaks lurk in all places where you cannot namespace things because they are not fully under your control, which can be IP addresses (big customers bring their own address spaces, and IPv4 address space is too small for generous segmentation), Domains, email addresses, phone numbers and so on. |