[charcircuit]

>and doing that will cause a huge one-time hit against your token limit if the session has grown large.

Anthropic already profited from generating those tokens. They can afford subsidize reloading context.

[pixl97]

No they can't, that's what you don't seem to get.

Reloading those tokens takes around the same effort as processing them in the first place.

It's ok to be ignorant of how the infrastructure for LLMs work, just don't be proud of it.

[charcircuit]

They literally can. They could make the API free to use if they wanted. There is no law that states that costs have to equal the cost it takes to process the request.