[rahimnathwani]

Yeah this is true. I could have sent the entire book up until that point as context. But doing that 100 times (once per chapter) would have meant sending roughly 50x the length of the book as input tokens (going from 0% to 100% as the book progressed).

This would be fine for a cheap model, but GPT 4.5 was not cheap!

I would have liked to have fewer, longer chapters, but my (few) experiments at getting it to output more tokens didn't have much impact.

[Aeolun]

Yeah, that’s what I eventually ended up doing. Quality and cost both went through the roof. To be fair, Claude is good about caching, and with a bunch of smart breakpoints, you pay only 10% for most generations.