[dartos]

Well the rate of new LLMs keep coming out, but since they’re all trying to model language, they should all be fairly interchangeable and potentially will converge.

It’s not hard for a product to swap the underlying LLM for a given task.

[ankit219]

I meant not a jump in text generation ability, but more like adding a completely new modality and the likes. With 4o, you can have a multimodal embedding space and provide more relevant context to a model for fewer tokens (and higher accuracy). Ideally everyone would get there, but upgrading your pipeline is more about getting the latest functionality faster rather than just a slightly better generation.

[dartos]

Well they did.

Then Google did.

Then llava.

The issue is that this technology has no most (other than the cost to create models and datasets)

There’s not a lot of secret sauce you can use that someone else can’t trivially replicate, given the resources.

It’s going to come down to good ol product design and engineering.

The issue is openai doesn’t seem to care about what their users want. (I don’t think their users know what they want either, but that’s another discussion)

They want more money to make bigger models in the hope that nobody else can or will.

They want to achieve regulatory capture as their moat.

For all their technical abilities at scaling LLM training and inference, I don’t get the feeling that they have great product direction.