[dimitrios1]

I found it just was not necessary for most cases. And the way I got there was working backwards from the "web scale" technologies like kafka, kinesis, and dynamodb.

I built a data ingestion system that handled an average of 300 messages/sec, peaking at 1,000, and writing to a single R3 RDS instance. You can do a lot by pushing simple scaling strategies to their limit. Everyone thinks they need to handle web scale, but really you just need to handle your scale.