[koyanisqatsi]

Elon used to be a software engineer so it's kinda weird how he thinks a legacy codebase like Twitter can be fixed up to use $1B less every year. Twitter is limited mainly by storage, their compute costs are negligible in comparison. So unless there is so much bloat in their hardware procurement process that they're over-provisioning storage by more than $1B a year there is no way to reduce costs. The hardware costs what it does and there is no way to rewrite the software at this point to reduce hardware costs and still maintain backward compatibility with the existing Twitter functionality (no matter how minimal the feature set seems to be for external observers).

So either this article is a lie or Elon is just making nonsensical statement to focus attention on what it costs to run Twitter at such a scale. If people don't pay then the functionality will continue to degrade and it's looking like there aren't enough willing buyers for $8/month. The only way Twitter can continue to operate is if the people that actively use the service start paying for it, every other financing option is now closed off and if Elon can't turn it around then I doubt anyone else can.

[Hydraulix989]

I struggle to find one correct remark in your comment. This take clearly suggests you have never worked on any similar infrastructure / distributed systems before at scale.

So you're telling me there's no backend or compute workers? And you're telling me that there are no opportunities for performance optimization? That they are already 100% native C++ services at full parallel utilization? That they've already tuned the size and cost of their ML models? That their model evaluation infra is fully saturated?

Storage is cheap. I believe they are on AWS, so hardware costs are already taken out of the equation. If so, even just moving to on-prem would save them the $1B easily.

[devoutsalsa]

> Storage is cheap

Seriously. They could back up all of Twitter on Backblaze for $7/month.

[indigodaddy]

Hah, I chuckled bigly, nice one.

[koyanisqatsi]

You should sit down and perform the storage calculation instead of struggling to find something correct in what I said because that will be a better use of your limited time.

[drewbug01]

> I believe they are on AWS, so hardware costs are already taken out of the equation. If so, even just moving to on-prem would save them the $1B easily.

They already run their own datacenters (not uncommon for companies started in that era). They have moved partially into the cloud for data science stuff - GCP, not AWS, according to their engineering blog.

So… odds are that there isn’t nearly as much waste as everyone wants to believe.

[unboxingelf]

DC is far more cost effective than public cloud for data intensive workloads.

[drewbug01]

Right, which is why they only executed a partial migration for certain workloads that make sense: https://blog.twitter.com/engineering/en_us/topics/infrastruc...

I think what really irks me is this meme that “obviously they have massive infrastructure waste.”

Does anyone have credible evidence that there is enormous waste in Twitter’s infrastructure? Just because Musk says “cut $1b in infra spending” doesn’t mean that there is $1b of things to cut, just lying about being wasted. Some of the smartest minds in our industry work/worked at Twitter. Are they that bad at their jobs?

What is actually going to happen is this: they’ll turn off all of their data warehousing stuff, blinding the business. They’ll cut their redundancies and backups, reducing the probability that they can do effective DR. They’ll reduce spending on observability so far that they won’t even know what’s going wrong (it’s surprisingly expensive). And that’ll get them to the $1b in cuts - by flying blind (both in a business sense, and a technical sense), and just hoping they don’t need to recover from a disaster.

[trowawee]

> If so, even just moving to on-prem would save them the $1B easily

Except you can't wave a magic wand and shift the entire distributed infrastructure for one of the five most visited sites on earth on-prem. Twitter has two DCs. Running their infra on-prem somewhere would require reworking the entire company from the ground up in a way that would take years and cost substantially more than $1b.

[indigodaddy]

Having their own DCs means they are on-prem already.

[hoten]

If they laid off 3700 people, assuming 200,000 dollars each, that's 75% of the way there.

edit: oh, I suppose you wouldn't consider personnel "infrastructure". I'll keep this up despite how off topic I am, it's interesting to compare the reduced cost in labor to the stated goal in reducing infrastructure costs. A billion here, a billion there... Soon we're talking about real money.

[koyanisqatsi]

I'm sure he can keep firing people and just keep the platform on life support but that's the same as admitting Twitter is going the way of Digg and all other social networks that tried to execute a pivot. The people that think he'll be able to turn it around don't really understand the scale of the problem.

[SOLAR_FIELDS]

There’s always a way to reduce costs. For storage specifically one could spend engineering time compressing data, storing it in a colder way, etc. The question is whether the trade offs, short term or long term, are worth it.

[koyanisqatsi]

I'm pretty certain that Twitter's code is essentially append-only. It's possible to add new features and keep the old stuff running but there is no way to re-architect it to reduce costs. They are already compressing the data so there is no improvement to be gained there. As for cold storage, re-engineering backups and other forms of redundancy will reduce long-term costs but it won't reduce their yearly operating requirements for storage and compute (storage fundamentally being the main limiting factor).

So either enough people pay to subsidize Twitter's operating costs for everyone else or it continues to get worse over time. There really isn't a 3rd option with better engineered software because there are no software engineers that can work on software of Twitter's scale and make it meaningfully more efficient in terms of storage and compute requirements.

Presumably we are on the brink of AGI, so maybe Elon knows something everyone else doesn't and he's just gonna use AGI to rewrite all of Twitter in assembly. /s

[indigodaddy]

AGI would be Artificial general intelligence?

[koyanisqatsi]

Yes.