[tyre]

So they're constantly hemorrhaging their most valuable clients?

Tech history is littered with the corpses of "open source but we sell hosting" services. Models are so expensive to train, you can't be losing the big clients once they get super profitable.

[MikeTheGreat]

This is genuine, noob question: how is this different from AWS?

I get that they're in very different businesses, but for both don't they have the issue that once a client gets big enough the client might decide to move the services in-house? Based on how much of the internet went down when that AWS data center crashed the answer is clearly "No" for AWS.

Is that because of physical, real-world infrastructure? Are there no open versions of their APIs? Is it too hard to migrate to something else once a client has achieved that size?

[NortySpock]

Data is heavy.

I would say "it's risky and requires a lot of labor to migrate without corruption, loss of data" and also minimizing downtime. Sure anyone can run pg_backup, but can you do it across 90 databases? Can you do it live? Can you coordinate rollout of the process, cutover, and monitor for failure? What's the cost of egress for this? Is the team your A-team or the B-team? Can you trust this to the B-team? Is it worth having this team spend all this time on a migration rather than, say, getting something new set up, or optimizing performance on an existing system?

I'm a database guy, but the same migration argument is presumably also extra work for (say) blob storage, networking, etc.

Since LLMs are stateless by their current implementation, switching to "the same open-weight model running in a different datacenter run by a different vendor" is "just" switching the API endpoint. (If they are the exact same shape, it's fine, if they differ somehow, there's perhaps some work to do there, fixing things and monitoring for failures on switch-over)

There are several open APIs it seems and OpenRouter.ai is doing a fine job making a commodity out of models and datacenters.

[ergocoder]

I don't think it's that difficult. Their servers are stateless too. S3 is easy to migrate.

Database is more difficult, but tons of people have done it successfully.... meanwhile people who host their own LLMs are relatively small in number in comparison.

Most companies don't do their own data centers mainly because it is more expensive and less reliable. It's something they can just pay for the problem to go away. The calculus for hosting your own LLM is probably similar.

Even Stripe who built their own coding agents and has tons of money/resources still decides not to host their own LLMs.

Still, many people will prefer open-weight models. It is similar to how we prefer linux but still use AWS/Render/and whatever. It doesn't lock us in, and we can move providers if we want to.

[Closi]

> This is genuine, noob question: how is this different from AWS?

AWS owns the hardware, and doesn't write a lot of the software.

AWS actually is kind of the opposite - it often takes open source software (e.g. Apache, Mongo, Kubernetes) and then makes money off it by hosting it itself (with some enhancements etc).

If they do develop their own software (e.g. with S3) they don't give away the source code so others can deploy it, as that's part of their secret sauce.

In this scenario, where they would be offering the open source model and then offering the same model hosted, there isn't really a moat here - they would be leasing the hardware from a company like AWS, and adding a margin, but it woudl be trivial for another company (or Amazon) to take their same model and offer it for the same price or less.