[steve_adams_86]

I accidentally got Zed to parse way more code than I intended last night and it cost close to $2 on the anthropic API. All I can think is how incredibly expensive it would be to feed an LLM text in hopes of making those connections. I don’t think you’re wrong, though. This is the territory where their ability to find patterns can feel pretty magical. It would cost many, many, many $2 though

[diggan]

> I accidentally got Zed to parse way more code than I intended last night and it cost close to $2 on the anthropic API

Is that one API call or some out of control process slinging 100s of requests?

Must have been a ton of data, as their most expensive model (Opus) seems to $15 per million input tokens. I guess if you just set it to use an entire project as the input, you'll hit 1m input tokens quickly.

[steve_adams_86]

Come to think of it, I’m not sure how Zed performs LLM requests with the inline assistant.

I wasn’t working in an enormous file, but I meant to highlight a block and accidentally highlighted the entire file and asked it to do something that made no sense in that context. It did its best to do something with the situation and eventually ran out of steam, haha. It’s possible that multiple requests needed to be made, or I was around the 200k context window.

Previous to this I’m fairly sure most of my requests cost fractions of pennies. My credit takes ages to decrease by any meaningful amount. Except until last night. It’s normally an extremely cost-effective tool for me.

[pcthrowaway]

This is a pretty good case for just using a local model. Even if it's 50% worse than Anthropic or whatever the gap is now between open models and proprietary state of the art, it's still likely 'good enough' to categorize a story in an old newspaper as missing from an author's known bibliography.

[steve_adams_86]

Good point. I use llama3.1 for a lot of small tasks and rarely feel like I need to use Claude instead. It’s fine. I’m even running the model a (big) step down from 70b, because I’ve only got 32GB of ram. It’s a solid model that probably costs me next to nothing to run.