Hacker News new | ask | show | jobs
by fillest 1429 days ago
If by "large" you mean e.g. "under relatively high load" then you can simulate this load and learn to deal with it. Populate your databases with a lot of generated data. Use load testing tools. Add latency and faults to different parts of the system (you can even inject errors into storage system (e.g. device-mapper)). At some point you even can rent some big servers (e.g. cloud) to test you problems and ideas - the cost of learning can be very effective. Make some checklists and iterate over all the components:

  - what happens if this stuff fails (and how to test it, how to analyze it)
  - what attack surface that stuff has
  - how to scale if it receives x10 requests or becomes x10 bigger
  - how can you explain to a new coworker how it works, how all the decisions were made
It will rapidly give you the real understanding and practice. System design is about dealing with problems, making compromises. It is very hard to make an effective book (probably even impossible - it should probably be an interactive course at least) that will walk you through this process, not just throw a bunch of "patterns" at you (which is not useful and can infect you with the shallow cargo-culting that our whole industry is so full of).