[socketcluster]

It makes me wonder about the state of their codebase if devs needs to consume more than $1500 per month.

It's interesting that AI is finally forcing businesses to think about coding maintenance costs though.

When I started working on https://saasufy.com/ as a dev tool many years ago, I was frustrated that no big company cared about software maintenance costs and I really couldn't imagine a world where maintenance costs would be a problem (which is what my platform was addressing). So this is one positive thing from my perspective, I guess. But how much longer before people put 2-and-2 together and realize that architectural complexity is the leading cause? That's the real moment I'm still waiting for.

Will what's left of the socio-economic system be sufficiently capitalist that I will be able to capitalize on that? That's my next problem.

[tsvetkov]

Why do you think the cap has anything to do with the quality of their codebase? Employees could've been tokenmaxxing for various reasons: learning, experimenting, trying to impress the management, ... Naturally, this leads to AI spending skyrocketing while the business value may not be totally clear. Which leads to caps being introduced to keep the budget under control and discourage/limit tokenmaxxing.

[socketcluster]

It's based on my experience as a software engineer who has worked on both clean and messy codebases with AI.

It's a very different experience with a messy codebase. In this case, the agent spends most of its time trying to gather the relevant context and it's like a game of whac-a-mole. The agent burns through tokens and can take a long time to resolve the issue with a lot of human intervention required. I would say it takes possibly just as long or longer than a human engineer would. Also, psychologically, the temptation for the engineer to trust the AI is massive because they don't want to load themselves up with all that ugly, complex context. They are more likely to let the agent create more hacks on top.

On a relatively well-structured codebase with loose coupling and high cohesion, the experience is usually very positive, mind-blowing, even; because it feels like the agent is reading your mind and fast-forwarding you. You don't need to correct it as much. And when you do, it's usually minor things.

The first case represents a net loss of value because tech debt is being added and compounding the complexity each time a problem is 'solved'. On the other hand, the second case is a significant speedup, for me, I would say it's at least a 5x speedup. I love using AI in this way. I'm in control and not at the mercy of the agent.

[tsvetkov]

I don't argue against the fact that codebase complexity increases token consumption on building context. My main point was that there are other factors affecting token consumption beyond just codebase complexity. Some of them may be related to engineering culture (verbose logs, flaky tests, lack of docs, weird hacks all over the place, etc.), some of them are organizational/social.

[socketcluster]

Sure. A lot of these things tend to go together. Weird hacks is a bad one. Those AI agents love to cheat and if they see highly elaborate hacks in the code, they won't hold back either.

[prymitive]

I have no idea how much I’ve spent, it’s invisible to me, the company doesn’t share it with me. I have no idea what “1 credit” means in terms of $$$, is that 1$? 0.1? 0.01? Is it even a fixed price? I have no idea how much will given take cost. Well, I can ask for a plan and extrapolate from that, but all perfectly reasonable looking plans eventually end up in a rabbit hole. Providers keep introducing new models and each is more expensive while offering modest improvements, it’s a silent inflation.

So I personally can easily believe that. Especially that a lot of people will just try to see if model can make that huge improvement / refactoring they’ve been hoping to do a reality, or tons of experiments to validate ideas.

[cactusplant7374]

If for each story the developer needs to fetch context for 10's of micro services I could see them using a lot of tokens.

[socketcluster]

True. I've worked on projects which required updating 3+ repos for each feature. Required carefully-timed staggered deployments.

It's often a sign of poor separation of concerns. Tight coupling and low cohesion.

On a good codebase with microservices, this should happen on rare occasions, but not every single time you add a new feature. Been there. Agreed those are particularly hard to work with using AI.