|
Let's take a page from graph theory, because I think it applies to large scalable systems as well. Some comments are alluding to the same issue from other perspectives, but I'm going to talk a bit about graph theory, because that's where I first encountered it. On a piece of paper, draw 5 nodes and make a fully connected graph. What shape does it have? Well, whatever way you drew it, the shape is quite recognizable and distinct. Now on a piece of paper, draw 1 million nodes and make a fully connected graph. You can use a computer if you want. What shape does it have? You can't tell, because all the lines are in the way? Alright, well what if we make 5 clusters, and represent them as nodes, and since it's a fully connecte graph, you can just connect those 5 nodes fully. It has a recognizable shape again! Nice :) There is the tiny little caveat that you now have a cluster of nodes represented as one node, but what could go wrong? More layers of indirection, that could go wrong. The more concrete stuff you have, the more you need to abstract away in order to maintain a high level overview, but the tradeoff is that while it is high level, it is less concrete (more abstract). |