[lenerdenator]

> How long will a SOTA model be necessary? If day to day work can be achieved on an open weight model, the most evaporates overnight.

Depends on the advantage it gives people and marketing of that advantage. You'd be surprised at how overpowered the average workplace laptop is. Each company I've been at has had at least some people who do non-technical roles using high-end hardware. Why? Because the account executive wants the fast machine and they get what they want.

You can apply the same to GenAI. Humans are notoriously bad at estimating actual needs when it comes to resource consumption. Best to have it and not need it than need it and not have it, especially if your competition just shelled out for SOTA.

And that's not even taking into consideration regulatory and customer concerns about where the AI you're serving requests with came from.

[zdragnar]

Execs getting what they want doesn't mean they let everyone have the same thing. It's far more likely that everyone else is using lesser equipment.

We have already seen tech workers at big name companies get whiplash from "leaderboards showing people using the most tokens!" as a good thing one month to being pressured to using fewer tokens a month later.

[lenerdenator]

> Execs getting what they want doesn't mean they let everyone have the same thing. It's far more likely that everyone else is using lesser equipment.

You'd be surprised. I've seen people request upgrades and get them. Not even execs, just employees. Even the average devices are probably overpowered these days. Chromebooks could do the vast majority of work in a corporate environment. Try getting workers to accept them, though.

Regardless, there are probably enough reasons to use proprietary Western AI for the time being, particularly in regulated industries, that poop won't completely hit the fan. Particularly if the CTOs start searching the phrase "Operation Aurora".

> We have already seen tech workers at big name companies get whiplash from "leaderboards showing people using the most tokens!" as a good thing one month to being pressured to using fewer tokens a month later.

Indeed. What do you do when you want a group of people to do something that they'd be otherwise adverse to doing at all? You rank them on it and let them get in a competition over who can do it the most. That's what tokenmaxxing was about: getting people to use the AI at all. Now that the people are using AI, we move onto another objective, which is getting them to use the tokens efficiently. The trick is measuring that efficiency.