Hacker News new | ask | show | jobs
by kridsdale3 642 days ago
Yes when I used to talk about this to interviewees, I described that every tool people commonly use is somewhere on the Big-O curves for scaling. Most of the time we don't really care if a tool is O(n) or O(10 n) or whatever.

At Meta, N tends to be hundreds of billions to hundreds of trillions.

So your algorithm REALLY matters. And git has a Big-O that is worse than Mercurial, so we had to switch.
2 comments
steventhedev 642 days ago
I'm gonna disagree with you there. The difference was with stat patterns, and the person at Facebook who ran the tests had something wrong with the disk setup that was causing it to run slowly. They ignored multiple responses that reproduced very different results.

Nail in the coffin on this was a benchmark GitHub ran two years ago that got the results that FB should have: git status within seconds.

Facebook didn't use mercurial because of big O, they used it because of hubris and a bad disk config.

link
sangnoir 642 days ago
> Facebook didn't use mercurial because of big O, they used it because of hubris and a bad disk config.

Half-remembering a blog post I read - the git maintainers also wouldn't give Facebook the time of day on code changes to accommodate FBs requirements. Mercurial was more amenable. This also disproves the "Facebook has a fork of evertyhing, because the attempted to upstream the changes they wanted)

link
deadmutex 642 days ago
This sounds plausible, but would love a source
link
steventhedev 642 days ago
I should probably just write it up into a post, but the git mailing list at the time is the source (I remember reading it from the side a few months after convincing our VP R&D to switch from svn to git). We were chuckling around the same time that FB had to reallocate the stack on Galaxy S2 phones because they were somehow unaware of proguard or unable to have it work properly with their codegen.

Anyways:

1. Github benchmark: https://github.blog/engineering/infrastructure/improve-git-m...

2. The original email thread: https://public-inbox.org/git/CB04005C.2C669%25joshua.redston...

3. There's another email thread that gets linked everywhere - but in light of the prior thread, the numbers don't track: https://public-inbox.org/git/CB5074CF.3AD7A%25joshua.redston...

I recall there being a message from someone either at AirBnB or Uber who mentioned that they have a similar monorepo but without the slow git status, but can't seem to find it now - it's likely on one of the other mailing list archives but didn't make it to this one.

Point being that painting this as "the community was hostile" or "git is too slow for FB" is just disingenuous. The FB engineer barely communicated with the git team (at least publicly) and when there was communication, it was pushing a single benchmark that was deeply flawed, and then ignoring feedback on how to both improve the performance of slow blame, commit by repacking checkpoint packfiles (a one-off effort) and also ignoring feedback that the benchmark numbers didn't make sense in absolute terms.

link
master_crab 642 days ago
If git is blocking you, you are using it wrong. Lotta instances of people treating it as an artifact repository. Use it correctly with a branching strategy that works for your use case and it's bulletproof.

Plenty of other customers with the same magnitude problems as Meta are using Git perfectly fine.

link
quicklime 642 days ago
Who are the others with the same magnitude as Google and Meta’s monorepos?
link
ec109685 641 days ago
Microsoft has all of windows in a single repository.
link
disgruntledphd2 642 days ago
Particularly in 2014, when the git thing happened.
link
master_crab 641 days ago
No one. Because people should know better than to use monorepos. Most teams at Amazon (which works at the same scale) religiously avoided them.
link
KaiserPro 642 days ago
> Plenty of other customers with the same magnitude problems as Meta are using Git perfectly fine.

I mean there aren't. there are perhaps three places that have the same scale problem.

A monorepo for a place with about 50k developers, that has been operating at that scale for 5 years.

The current checkout if not sparse would be >80gigs

The commit rate is > 20 a second.

no amount of branching strategy is going to help on that.

I love git, I used it professionally since 2010, but git is not a good fit for something _massive_

link
ec109685 641 days ago
Microsoft has made git work for massive repositories through a lot of blood sweat and tears: https://github.blog/open-source/git/the-story-of-scalar/
link