[teaearlgraycold]

What kinds of tasks are you giving to the LLMs? I don’t think I do anything that can rack up costs like that. I can only imagine you’re using lots of instances simultaneously. I’d love to know more and ideally see the deliverables if the code is public, or even just the product.

I recently talked with a guy that’s pretty smart and is building a good product with a clear market. I understood the idea and encouraged him to ship.

Then he showed me what he considers his real work, and went off on a madman’s raving presentation of some supposedly hyper scalable revolutionary encrypted block chain dApp agentic operating system. He was building all of this using lots of agents. And I could totally see him spending $500/day on tokens. But I also couldn’t get him to explain the use case. I couldn’t imagine one myself. I’m by default suspicious of large AI bills. People usually only have a short amount of roadmap that’s well thought out and proven. Building faster means you just hit a new bottleneck in product design.

But I want to learn more and accrue more case studies of AI use in software engineering. Sometimes I hear of some really great software engineering techniques only possible thanks to AI (stuff like running 3 models in parallel optimizing a hot loop, comparing outputs with a rigorous test suite and fuzzing).

[denimnerd42]

We don't spend that much money every day but here's the gist: We have a distributed system that has several components that don't meet the performance requirements of the next uplift we need to do. We need to carefully consider the tradeoffs of things like how to shard a few of the databases, how to rearchitect the ETL flow that comes off the system and is used for analysis. We think of a few approaches and then we get the coding assistants to blast through the end to end development of each approach discovering all the known unknowns and unknown unknowns along the way. Then we can load test each method, profile them, analyze them manually and with the LLM. Then we can pick the solution and take another shot at implementing it with the coding agent, but more carefully and with more oversight with all the things we learned.

We don't hit those high numbers every day. An average day is $50-100 max.

As far as home projects. Something like write a GUI desktop or phone application from scratch. The LLM has to reference a lot of code and API docs to figure out what to do and spends a lot of time thinking while debugging. It gets expensive :/