[apohn]

It's possible to have teams that save millions of dollars a year and not be worth it to keep them.

For the sake of argument, let's say a statistics team has 5 people.

Cost of Employee at FB, including insurance, office space, 401K match, salary, bonuses = 250K/year (probably very conservative).

Cost of Data and Software Infrastructure to support them (including people to respond to Infrastructure support tickets), let's just be very conservative = 100K/year.

Cost of People Management overhead to support them. Includes salary of at least one manager, not to mention the time of a program manager, project manager, product manager, or whomever else. Let's just say 500K/year.

Total = 1.85 Million/Year.

Let's say this team of 5 people comes up models that save the company $4M a year. I once had a VP tell me that to justify a Data Scientist on the team, they needed to have a savings of 10X what they cost the company to have that person on staff. I know this logic and math seems very weak and hazy. Mapping costs is a strange thing. But this is how some decision makers think, and this is how people get cut.

[ericlippert]

The team was all mathematicians. We did the math. I helped one of our data scientists put a model into production that saved $15M a year from that model alone, and we had a dozen people like that. We were working on signal loss models that had potential to save billions. I genuinely do not understand the logic of cutting this team to save costs.

[acqq]

Eric, my best wishes to you, I've also enjoyed reading your texts, at these older times when you were allowed to write about your work.

Having had some similar experiences to yours now, I don't believe there has to be strict logic behind the managerial decisions leading to big changes. That's not how they are made, and that happens more often and with more impact than we typically register in our own environment, as we are busy doing our specific tasks. I know that it can sound cynical but I think it correctly reflects the reality.

In one specific case from my previous work, I know from those present where the decisions were made, that a decision about hundreds of people working further of not on many running projects was made after one high manager left and the few remaining who were the only one deciding literally had a short talk: "OK, who wants to take over these, I won't, do you?", "no", "no", "me neither." "OK, then let's dismount all that." And so it went. And similarly, it's not that it was not profitable for the company, it was clearly documented. The decision of each of those involved was then explainable with "it didn't match our vision of where we want to concentrate our company's effort." It is sometimes as simple as that. The "high managers" so often score additional points whenever they decide that the company makes less of different stuff.

Steve Jobs was, of course, famous for abandoning different projects in Apple on his comeback, and it provably gave the results. But I also see the companies overnight losing the proficiency in some fields based on managerial decisions impulsively made, performing even worse later. I don't have any grand narrative based on these experiences to push, except to state my belief that sometimes the "reasons" are extremely simple and very, very mundane, to the point of causing huge disappointment to those who heard so many decisions presented as strictly a result of precise measurements and deliberations, who knew they did their best and were aware that "nothing was wrong."

It does leave one questioning why they correctly invested as much energy in what they did, and if they made right decisions during these times, from a newly obtained perspective.

[apohn]

>I genuinely do not understand the logic of cutting this team to save costs.

I've been in a situation where a company was under pressure, was trying to make a big pivot, and there where multiple rounds of layoffs.

At one point I could only make sense of it by picturing a somewhat blind lumberjack getting an order that says "There's a forest that needs 15% of trees cut. Go cut." Good trees get get, bad trees get cut. Thankfully we are not trees and if we get cut we can move on. We don't die just because we got chopped down.

[drc500free]

Unfortunately, top-down mandates are imperfect and should be avoided as much as possible. Net profit matters to an operator who cares about today's profitability, but not at all to someone whose paradigm is "thinking in bets" and future payoffs. And the street has been rewarding people who ignore today's profits in favor of the narrative about tomorrow's growth.

From afar, it looks like Meta's leadership is a bunch of future thinkers who got told to cut today's costs, and it's not a well-practiced muscle for them.

[lurker616]

Perhaps in the future the company would not be adding any new models or require optimization of any new surfaces, so they don't expect to be spending enough on new initiatives to justify optimizing them. And all the existing initiatives have been optimized efficiently already (though that does seem unlikely when I type it out).

[hkon]

> We were working on signal loss models that had potential to save billions

What are signal loss models in this context?

[ericlippert]

"Signal loss" is the overarching term for all the factors that lead to the company being less able to make good inferences about users. Not just the obvious consideration of "how do we serve an ad that is relevant to the user?" but for any data-driven decision that affects a user's experience.

The biggest recent cause of signal loss was Apple changing the rules for apps on their phone, but there are plenty of other causes.

The idea of a signal loss model is to identify ways to work around signal loss and still do a good job of making a decision with the data you have, when some of the data you were relying upon disappears suddenly.

[bombcar]

Perhaps an example would be - we no longer have location data for users, but we do have time of activity, so we can presume that during daylight hours in the USA most of our activity is coming from there, things like that.

But with more inputs and such.

[drc500free]

These employees are making a lot more than $250k just in base salary. Cost is probably closer to $1M each, all in. "A few million" in net cost savings isn't much for a team that probably costs $5M a year.

It would definitely be better to find another internal home (assuming the team is portable without its mother team that got cut), but sometimes these decisions are made quickly without a lot of granularity. They aren't necessarily going to find one sub-team that saves only ~1x their cost in net profit and figure out how to transplant them to another org.

He seems to have taken away the important lesson - if you're not primary you're in danger.

[vikingerik]

How in the world are you getting from $250k salary to $1M total cost? Stuff like office space and equipment/services, health insurance, HR overhead are constants per person, they don't scale up with salary. Are you assuming that some big bonus or grant package is necessary?

[drc500free]

Yes, their total comp is $500k+. They are taking up a portion of the management time of someone whose total comp is approaching (or over) $1M.

Software Engineer: https://www.levels.fyi/companies/facebook/salaries/software-...

Software Engineering Manager: https://www.levels.fyi/companies/facebook/salaries/software-...

[TheNewsIsHere]

The trick with discussing any numbers like this are variables that none of us can know without more intimate knowledge of a firm. For example, my spouse works for an SV firm. His team is 100% WFH, 100% of the time. They have no permanently allocated office space in any of the company’s buildings anywhere in the world.

However, they’re paying out bonuses twice a year, annual (PB)RSUs, (specifically for us) around almost $30k/yr in employer contributions to health insurance and our HSA combined, music streaming subscription, and so on.

The benefits, the bonuses, the extras, they all add up and are all very company specific. I’m not saying you’re wrong by any stretch. But with the number of extra benefits, healthcare, and everything else that’s different from employer to employer, we are all just guessing.

[johnfn]

Facebook employees make a lot more than 250k. Someone with Eric lippert’s level of experience probably makes well over 600-700k in total compensation - just see levels.fyi!

[athrowaway12]

A big portion of the comp especially at higher levels is in stock grants... and Meta stock just dropped 75% in value this year.

These grants are valued at the market price at time of hire (or refresh).

So maybe pre-2022 the comp was 700k...

[trenchgun]

It was claimed to be millions of net savings per deployed model.

[AlwaysRock]

Doesnt this paragraph indicate that they were making millions of dollars in savings OVER their cost of operating?

"The PPL team in particular was at the point where we were regularly putting models into production that on net reduced costs by millions of dollars a year over the cost of the work"

[kqr]

If the company as a whole has an ROI of 10×, then a position with a 4× ROI is actually reducing the marginal profit of the company.