[chihuahua]

It's amazing that it took months to figure this out. "Well we thought that if engineers are told to maximize costs through AI use, to consume as much as possible of a resource that costs us money, then obviously good things will happen. Imagine my surprise when it didn't turn out that way."

Imagine if engineers were ranked based on their AWS spend. People allocate VMs and fill databases with terabytes of random bits, to get to the top of the AWS leaderboard. If you don't do this, you're ranked at the bottom, and good luck at the next review cycle. Who could have expected that this is not the road to success?

[this_user]

The point of this was always to explore what is possible with AI as quickly as possible. Obviously, there is going to be a lot of waste, but the 5-10% of employees who are truly thinking about it and discovering novel applications are what you are truly after. Because right now, you effectively have a giant, as of yet poorly explored space of potential uses.

Anyone who can find the actually valuable portions of the space early has a potentially huge competitive advantage. Even if the result of the experiment is the negative that AI is actually mostly not that useful, that is still extremely useful information in a time of great uncertainty regarding outcomes.

The bottom line is that this approach may be expensive, but if you have the money to burn, it's far from the worst strategy if you are trying to position yourself correctly for the future.

[uejfiweun]

The thing I don't get though, is that most people just don't have that much work they need to do. I can use AI to pretty easily get my work done just via the regular chat interfaces. But because of the tokenmaxxing metrics that leadership tracks, I end up just having the AI deliberate for hours on random things just so that I can boost my token numbers. I think tokenmaxxing for the end goal you described is only realistic when the engineers are truly buried under a backlog of work.

[smoofles]

Not being buried under a backlog of work is one aspect, and the other is that the sheer _urgency_ of these efforts makes it look like companies like Uber could be displaced in a year or two by someone who gets lucky with AI use.

Which absolutely isn’t the case. Even if someone would manage to overtake a market leader on tech merit alone, within 1-2 years, thanks to AI, markets don’t swing on such short notices. The fake urgency is absolutely psychotic.

[expedition32]

Ha the real stars of Uber aren't the programmers. It is the lawyers.

[adrianN]

What’s the huge advantage though? Adopting workflows that give big productivity gains is relatively easy even for big corporations. It’s only an advantage if you can keep it secret.

OTOH maybe we’re in for a future of patenting prompts.

[Bjartr]

> The point of this was always to explore what is possible with AI as quickly as possible.

If that was the intent, the messaging at many companies failed to communicate that. The message was "increase this metric", not "explore this space".

[davnicwil]

I think unfortunately it's not about what seems obvious, or even what seems more likely, but about what seems retrospectively justifiable regardless of outcome.

The incentive structure of this type of decision is 'absolutely under no circumstances existentially mess up'. Ostensibly with respect to the organisation, but in actual reality much more so with respect to the individual(s) involved in the decision.

If everyone else is doing something that kind of obviously makes no sense, and you decide to break from the crowd by instead doing what does make sense, then there's a pretty solid chance of gaining a temporary edge while reality resolves the truth. But those gains probably won't matter all that much for the organisation, or indeed your position within it. It's a solid chance of an unimportant gain.

However on the other hand, there's a tail risk that something very unexpected happens and the thing everyone's doing that makes no sense actually turns out to make sense - sometimes even for entirely unpredictable incidental reasons - and then, well, you're in trouble. Not necessarily 'you' the organisation.. they'll likely be able to catch up and it won't matter that much. But for 'you' personally, the decision maker, it's very much not good.

As a bonus, in the much more likely scenario that the thing that makes no sense turns out to indeed make no sense, you're in the same boat as everyone else, there's no relative loss, and most importantly you don't stick out as someone who did something as risky as to go against the prevailing, albeit pretty clearly nonsensical, sentiment.

So basically, game theory tells you pretty quickly to just go with the thing that makes no sense if you're optimising for some (weighted) cross of what's best for the organisation and yourself as the decision maker.

[saghm]

Someday maybe Goodhart's Law will be intuitive to people making decisions like this, but not any time soon I guess

[roxolotl]

The inability of leaders to understand Goodhart’s Law is always a sight to behold. They see a number go up and pat themselves on the back for how well their employees are making it go up without ever wondering if the thing they care about is happening.

[pmg101]

This is one explanation, sure.

Isn't it more likely that they simply don't in fact care about the "thing they care about", only the metric?

They can plot the metric on a chart and receive praise, so that's what they're interested in.

[roxolotl]

That’s an even worse characterization of them isn’t it? They don’t even care about the end result just the metric. That would take them from clueless to malicious.

[pmg101]

I wouldn't describe it as malice. If your job is to make the line go up, you make the line go up, and are rewarded for doing so, then you have done your job.

[dgellow]

> It's amazing that it took months to figure this out

We aren’t there yet, so far it is just a COO questioning the investment

[solenoid0937]

You say "amazing that it took months to figure this out" as if the answer to the question is obvious.

But it's not. Some FAANGs are doing amazing things with unlimited tokens. Other companies have no clue what to do with tokens, they've just told their engineers to max them.

It really depends on how you're using the tokens. If you're just using them for Codex and Claude Code - yeah, tokenmaxxing is incredibly dumb.

[SecretDreams]

Show me some fang that have made nice outwards facing products through a fully embraced AI workflow?

AI is an accelerator that engineers should know and have access to, but it's not something that should have mandated usage and quotas around. It's also absolutely dangerous for young engineers and the like - it fundamentally denies you of the "learning" aspect. I'm now seeing in interviews young graduates being given AI tasks to complete and they come back with a correct solution and no concept of how it is working.

You learn and reinforce learning by DOING and reading in depth. High level summaries don't teach anything and are the kinds of things only VPs care about. So, unless the intention in the future is for everyone to be a VP using AI to do the work, we need some middle ground here and some real thought around implementation of these tools or there's going to be a generational canyon gap of knowledge between being able to "say" and being able to "do".

[saghm]

In other words, people who are productive get more done when you scale up what they're already doing, and people who aren't productive will not magically become productive when you scale up what they're already doing. That's incredibly obvious, because we've seen how this plays out repeatedly in so many different ways (lines of code, commits, tickets closed, etc.), and it has nothing to do with tokens or even programming, but just how trying to manage people works.

[steveBK123]

> Some FAANGs are doing amazing things with unlimited tokens. Others have no clue what to do with tokens.

Unlimited tokens is different from “use AI a lot or we will fire you, and we are counting token consumption as usage”. Obviously the latter is stupid and yet it was done in many places.

[SpicyLemonZest]

I'm not convinced it actually was done in many places, although I understand why in a bad job market people don't trust that it isn't happening in secret. Every time I've heard of a token leaderboard or such it's come with a denial that the company is using it as an employee performance metric.

[steveBK123]

> it's come with a denial that the company is using it as an employee performance metric.

Surely the anonymous employee feedback polls are totally anonymous too. BigCorp loves you, is family, and would never harm you.

[fsloth]

> Some FAANGs are doing amazing things with unlimited tokens.

Would love to know what things!

[arkadiytehgraet]

OP (solenoid0937) is an unfounded AI-hype peddler and an Anthropic shill (check their comment history), do not expect them to provide an actual example of their wild claims.

[dgellow]

I checked their history and they seem alright ¯\_(ツ)_/¯

[arkadiytehgraet]

Here, just a few examples over the last 9 days, in no particular order of exaggeration magnitude:

Unsubstantiated claims of 3x-10x productivity at allegedly a FAANG: https://news.ycombinator.com/item?id=48174666

More claims of the same, still unsubstantiated: https://news.ycombinator.com/item?id=48173995

Even more claims of the same, still nothing of substance: https://news.ycombinator.com/item?id=48173975

Outlandish claim of using 300$ in tokens in 1 hour at allegedly a FAANG: https://news.ycombinator.com/item?id=48158438

Ostensibly great things with unlimited tokens at allegedly a FAANG: https://news.ycombinator.com/item?id=48269185

Complaints about HN "not getting" AI: https://news.ycombinator.com/item?id=48245673

An attempt to validate vibe-coding in production: https://news.ycombinator.com/item?id=48243651

Whitewashing Claude degradation: https://news.ycombinator.com/item?id=48211638

More Claude degradation whitewashing: https://news.ycombinator.com/item?id=48245784

Whitewashing Anthropic: https://news.ycombinator.com/item?id=48199951

Promises of coming AI revolution: https://news.ycombinator.com/item?id=48188104

All of this serves to hype up the LLM technology with absolutely outlandish claims, while also propping up Anthropic online.

[solenoid0937]

This is so cool, I have my own fan on HN that cherry picks my comment history to call me a shill!

[kbelder]

He may not be a shill, he may just be terribly and enthusiastically wrong. Lots of HN posters are.

[morpheuskafka]

> But it's not. Some FAANGs are doing amazing things with unlimited tokens

Giving someone unlimited access to a resources is not the same as directing or incentivizing them to use it for the sake of using it which is what the parent comment criticized.

As for the other FAANGs, Meta and Google have (not good but still) frontier models of their own, so they are very different from a company paying API costs per token.

[dgellow]

Where can I see those amazing things done by FAANGs?

[solenoid0937]

Join one!

[dgellow]

Anything more concrete?

[arkadiytehgraet]

You got baited by an LLM-hype peddler and Anthropic shill, do not expect anything of substance. See https://news.ycombinator.com/item?id=48272695