Hacker News new | ask | show | jobs
by tikhonj 1088 days ago
There's a rough rule of thumb that you should expect to redesign your system to handle each order of magnitude increase in scale, and I figure it applies here too—gracefully handling that size of repo would require substantial engineering work, and they have plenty of time to handle it before human-oriented open source repos get even close to the current limit.
2 comments

I'm not sure redesigns were necessary between going 1 to 10, from 10 to 100, from 100 to 1000, from 1000 to 10'000, from 10'000 to 100'000, or from 100'000 to 1000'000 which we're now at. It sounds like a sensible engineering rule, but I'm not sure it translates to software, or at least not in this case. I don't know of any design changes made to Git since it was first created, there's no v1 and v2 repositories for example.
It depends on how quickly you pass through each order of magnitude milestone. I remember reading about how MySpace grew something like five orders of magnitude in less than a year, and no matter how scalable your architecture is you're going to hit a point during that where you need to rearchitect your whole system.

Slower growth allows for forward planning and incremental architectural changes.

> there's no v1 and v2 repositories for example

We wouldn’t know. GitHub is probably running something very different to normal local git including optimizations for performance and cost.

They must only ensure API/protocol compatibility and could have already replaced everything else many times over.

> There's a rough rule of thumb that you should expect to redesign your system to handle each order of magnitude increase in scale

I rather know the rule: by good engineering, you can modify a system to handle a one magnitude increase with respect what it was designed for. As soon as a two magnitude increase can occur, you better redesign the system.