[pornel]

which is why they charge extra for an option to hold >256K tokens.

The cost may be smoothly variable, but likely there's a bimodal distribution of users who barely use any context and users who push it to the max. Average price across both extremes fits nobody, but averages per kind of workload can be close enough.

Having it as a separate model makes it easier to load-balance the traffic.