[bs7280]

I have ancedotal examples of claude code choosing a solution to a problem that is ridiculously token inefficient.

One example - was giving several agents different sub problems to solve in a complex ML / forecasting problem. Each agent would write + run + read a jupyter notebook. This worked ok, the notebooks would be verbose but it was fine... until one of them wrote out hundreds of thousands of rows to a cell output, creating a 500MB ipynb file. Claude tried several times to read it and it used my entire context limit.

The solution was to prescribe a better structure of doing the world (via CLI analysis scripts + folders to save research results to). But this required some planning, thought, and design work by me the operator.

When I see people spending $10k a month in tokens, I can only assume they are taking lazy hands off approaches to solving problems with the expensive hammer that is claude code. EX: have claude read all your emails every day... the lazy solution is to simply do that, but a smarter solution is to first filter the email body HTML to remove the noise.

[52-6F-62]

> have claude read all your emails every day

But that is exactly what it is sold to people to do as a panacea: consume all the data, produce insights.

Nobody is being instructed to be judicious. Everyone is being instructed to use it as much as possible for all problem areas.

[at-fates-hands]

>> Nobody is being instructed to be judicious. Everyone is being instructed to use it as much as possible for all problem areas.

Do you think this is because the LLM owners have such a massive ROI they're trying to cover so they're actively encouraging teams not to be judicious so then you get into this vicious cycle where both the LLMs and companies are both burning through cast like crazy?

[bs7280]

I should of emphasized it better in my comment but, the nuance of "read all my emails every day" as a prompt can yield a wildly different solution than "read recent emails every day". The first may literally read all your emails over and over, which is a ton of tokens for little gain. The latter is orders of magnitude fewer tokens with the same if not more productivity.

The difference here is just one word in the prompt, but serves as an example of how just a little but of deliberate thought in one's prompt can yield massive efficiency in outcome.

Whats wild to see both online and at work, non engineers given vibe code like tools will quickly show their ignorance to the importance of deliberate design and need for specific instructions one learns via coding. The "missing semi colon" meme is an example of the intuition we all developed early in our coding careers.

Many people are hoping AI can build and design for them, when in reality the deliberate design choices up front are as important if not more so than before AI.

[hirako2000]

If you make 500k and aren't spending 250k in token, you should get fired.

[quikoa]

I can't even tell whether this comment is serious or not.

[ori_b]

https://www.businessinsider.com/jensen-huang-500k-engineers-...

[ryandrake]

Person who profits indirectly from AI token spend advocates AI token spend.

[marcosdumay]

Nobody would invent something like that as a joke...

[bostik]

> have claude read all your emails every day...

To be fair, I do that. 2-3 times a day, in fact. Not all of my emails (the archive has ballooned to several hundred thousand messages total), but the most recent ones certainly.

My standard prompt is along the lines of "go through the last N days of my emails, identify all threads that I need to know about, action on or follow up with". N is usually a number between 2 and 5. I've specified a standing of set of rules to easily know what is likely a source of noise to aid in skipping the bot spam.

The company is charged API pricing through an enterprise contract, and I remain persistently curious how much I burn. My daily admin-related token expenses appear to fluctuate between $1 and $5. For something that saves me up to 2h of time a day I consider that a rather tolerable deal. (When I dive in to code to do refactors or deep investigations, I can spend as much as $25 a day.)

[bs7280]

This is a good example of doing it in a deliberate way that absolute is worth the tokens etc... especially when you are keeping tabs on the cost vs time saved.

The example I was thinking of would be a vibe coder having it "read my emails every hour" only for claude to read the same 1000 emails over and over...