[cmrdporcupine]

"Feedback from these engineers ultimately culminated in the managers making the decision to give automatic contributor rights to any Facebook engineer working on Presto, so that these engineers could move faster."

I'm confused -- does Facebook not have similar code review infrastructure as Google, etc. That is: nothing -- I mean nothing -- gets "contributed" at Google without it going through code review.

Were FB engineers able to commit directly without review? Or is it that they were given some kind of "owners" privilege to fast track reviews? This all sounds bad.

[bitsondatadev]

Facebook has their own internal reviews for their own internal projects and forks of open source projects. I think the issue came down to Facebook adding anyone working on the project to have the ability to merge code to the open source project.

Typically if a company wants to contribute to open source, you have to create a PR and have it be reviewed and merged by the maintainers of the open source project on top of internal review. Facebook management decided to circumvent that process.

[radicality]

I spent many years at FB, but no longer there, but that doesn’t even sound like a ‘special case’? Every engineer at Fb has access to pretty much all the code, and after an internal code review for your code change, you can get it committed to the internal repos. For open source things, something syncs the external and internal changes ocassionaly I think. I imagine the problem here was that now external people can’t comment on internal PRs etc.

I even remember some of my internal commits making it into FBs open source code on GitHub, even though at the time I had no idea that this specific area of FBs codebase is open source.

[billjings]

I worked at FB, too, and I can confirm you speak the truth.

This process is problematic, though. The "special case" is the standard process at FB: to make external contributors second class citizens.

They have their reasons, but doing this pushes the locus of discussion and action to the place where work happens fastest: inside the company. That means that the interests of FB engineers drive the project; in other words, not open.

That argument is I'm sure open to some logical nitpicking. But the evidence speaks for itself: Facebook open source projects aren't responsive to the outside community, and they language when FB's priorities shift. So their strategy has been pennywise, pound foolish in my view: they get the short term benefits of an "open" project, but they're incapable of actually being good open source custodians.

[natebc]

> They have their reasons, but doing this pushes the locus of discussion and action to the place where work happens fastest: inside the company. That means that the interests of FB engineers drive the project; in other words, not open.

As an outsider and frequent consumer of some high profile open source projects 'owned' by Big Tech Companies I get the feeling that this same thing happens everywhere.

It probably does make good business sense but from out here it's a little odd.

[billjings]

Not all of them. Square doesn't quite meet the bar for the top tier, but I can confirm that nobody gets special treatment for the open source work I'm involved with. It's easier to get face time with maintainers, sure, but you have no special access to the external project.

[zeroonetwothree]

It might not be pound foolish to the company. Their primary goal is making money not fostering open source.

[billjings]

I mean, they chose to invest in making this an open source project. And now it's been forked, which requires them to either forego community contribution, or migrate.

Remind me again what the win was here? It seems like they would have been better off not open sourcing it in the first place.

[bitsondatadev]

That’s exactly the point of the post too. Sometimes business and open source align and other times they do not. Trino was created as the project and community the puts community and individuals that contribute as the owners.

It’s not bad or evil or wrong that Facebook does this…it just is.

[nl]

Often (usually?) it's not about money though.

Say you work at FB on unannounced project X. You want to integrate with open source component Y, from a FB open source project but you have questions about the right way to do it, and how to get needed features into the open source project.

You can't discuss this in public channels, and all the maintainers are right there available to you internally. So what do you do?

[sorry_outta_gas]

His end goal is even worse In the early days of facebook Mark Zuckerburg would wander into the company bathrooms and if he noticed someone sitting down in the stalls he would pop his head over and try to talk to them about their projects. Or if he was taking a poop he would host an emergency meeting and he would tell them to come over and pop their head over the stall to talk it out.

Everyone just went along with it because it was either YOLO SILICON VALLEY LMAO or they were just too intimidated.

That all stopped when Michael Moritz, legendary silicon valley investor, and one of Facebook biggest early investors and shareholders, was at the campus doing research for leading a 2nd round of funding. He was doing diligence all day and at one point had to poop and that's when Zuckerburg popped his head over with a smile to ask how's the diligence coming along.

Michael Moritz, not one to mince words, was apoplectic. 'GET THE FUCK OUT HERE YOU IDiiOT LIZARD LOOKING FUCKER.' Mark Zuckerburg nervously tried to laugh it off and persisted, because he really loved intimate poop conversations 'Aw c'mon Michael, it's silicon valley'. Zuckerburg then withdrew after Moritz flung his cellphone into his eye socket.

30 minutes later, Mark was in a very import meeting (where he banned questions about his black eye) when Moritz walked into the conference room.

'Everyone except Mark Zuckerburg, OUT'. As intimidated as they were of Zuckerburg, at the time Moritz was the bigger deal, and they all scurried out of the room.

Zuckerburg, however, is not one to be intimated by anyone. Not the Winkewoz twins, not Eduardo Savarn, not Peter Thiel, and not one of his biggest shareholder Michael Moritz. Zuckerburg passionately defended his practice, but Michael Moritz was having none of that. Moritz told him that it was a ticking PR and HR nightmare, and threatened to pull out of leading the 2nd round of funding if Mark continued, which would have been a catastrophe for the company. Zuckerburg pretended to arbitrate 'Ok fine, but you need to give me a good reason, because if it were normal, there would be no problem'.

Moritz was flabberghasted at this response. Was this a serious question? He answered with the most obvious answer 'Because.... it's not FUCKING NORMAL'. Unknown to Moritz, Zuckerburg had guessed a conversation like this would happen as soon as he was kicked out of the toilet stall, and began formulating a strategy to counter Moritz demands. Zuckerburg knew that Moritz would have all the leverage, but Zuckerburg was a master strategist.

Zuckerburg went for the pounce. 'Okay, I'll lets write out an agreement, in writing I'll rescind the policy because it's not normal'. Moritz was dumbfounded, but he was used to being dumbfounded by eccentric tech founders, afterall he was also an early investor in Apple, and he still found Zuckerburg tame compared to Steve Jobs. Moritz had a long day of work so they signed the agreement so that he could go back to doing his due diligence.

When Moritz left, a broad grin spread across Zuckerburg's face. " 'Not Normal' eh? " Zuckerburg said with a menacing laugh. Ever since then, Mark Zuckerburg has been on a life-long crusade to normalize poop conversations.

He had a checklist of what he needed to accomplish in order to realize this. His advisors would tell him it's impossible, but one by one Zuckerburg checked off the list. From normalizing smart phone use on the toilet (actually a collaboration between Mark Zuckerburg and Steve Jobs), to trusting Mark with their private photos, to normalizing people giving up their internet browsing privacy.

In 2015, Zuckerburg knew he would hit a wall, having people watch you while you poop was still too much of a leap. That's when Zuckerburg decided to buy Occulus, and eventually shift his company towards virtual reality.

If he could coax people into having life-like conversations while they were pooping in a virtual reality, then doing it in the real world wouldn't be too big of a leap.

Do you read facebook or instagram while you're pooping? Ever consider what urges you to do that? It's not your personal preference, it's by Mark Zuckerburg's design.

Zuckerburg only has 3 more boxes to check off before poop conversations are normalized.

Mark Zuckerburg wants to watch you poop. Are you going to let him?

[tomcam]

That’s… that’s fiction, right? Because if so, bravo, that was hilarious. If not… Please Lord, tell me it’s not true.

[GrumpyNl]

Nice story, but whats the source of this story, only Zuckerberg or Moritz can have told the story.

[kmonsen]

Facebook/Meta does not have owners, so any two people in the company can land a change.

[benreesman]

And one engineer can (or could in 2018) put the unix name of the person sitting next to them in the "Reviewed By:" field and force push something, but it isn't/wasn't The Done Thing outside of the most extreme 3am SEV-0 scenarios.

Any two people are in a less-extreme but similar boat: if someone puts up a bad diff and someone without a stake in the code accepts it, they had better hope to hell nothing goes wrong.

SEV review is a remarkably enlightened process for what it is, but you do not want to sit there explaining to extreme-seniority people why you YOLO'd something into Presto without buy in from a Presto hacker with your manager sitting behind you already thinking about how much this is going to get harped on in calibration.

The `OWNERS` file at FB is in `hg log`, but it's there.

[eru]

If I remember right, Google also had a mechanism for pushing something through outside of normal review. But it was something more disciplined than just faking a review by putting your co-workers name in a reviewed-by field.

Because it was more disciplined, those commits could be automatically marked for later review.

[zeroonetwothree]

I believe you cannot force push anymore.

[danbrooks]

Agree. Wasn’t aware of force push (2021-2022).

[benreesman]

That is probably for the best.

In the bad/fun old days of extremely small on-call loop, a less clear/organized escalation chain, and fairly "roll your own" per-team tooling, once in a blue moon a whole business unit would be down because 10k machines were crash-looping and you just needed `if (NULL == ptr) return;` on like one line to end the carnage. There are exceptions to every rule I suppose.

But in 7 years at FB I think I force slammed someone else as reviewer with merely their verbal approval in the middle of the night 2 or 3 times, and never after like, 2015.

[DannyBee]

One can imagine that auditors were not particularly enthused about this :)

[benreesman]

Oh anything on user data or money or illegal content shit or anything like that is a whole different ballgame.

They mail you the shit off your desk for playing games with any of that stuff, and it’s thoroughly, thoroughly checked.

[avalys]

For a long time, it was worse than that - once a change was "accepted", you could amend the commit with changes to literally any other file in the entire monorepo, and land it without any further review.

They finally added a "final review" step to ensure that someone eventually takes a look at these changes after the initial accept, but that still occurs several days after the commit.

[toast0]

Can you still add

Reviewed-by: yourself

in the commit message to skip review? That was my favorite one.

[dsundstrom]

We ran Presto directly on Github (I think it still does), so everything went directly to the project without a private internal review first. We designed it this way, so that everyone in the community could participate in the full process.

[micromacrofoot]

There's code review, and there's "code review" where there's a deadline and someone says "can you click approve on this real quick," which is essentially a direct contribution with a compliance dance. I've seen it at large companies like these on several occasions.

[zip1234]

As far as I know nobody can just make changes unilaterally without another engineer's approval.