[k__]

As I understand it, they would have to train a whole knew model to hard cap it's context to different lengths. That would be cheaper to train and had cheaper inf, but still a huge investment.

So I'd guess it's API level.

[gpugreg]

From the Kimi K3 technical report:

    Kimi K3 supports a context window of up to 1 million tokens. We achieve this through extending
    the context window progressively as training proceeds, following a four-stage curriculum. The
    window grows from 8K to 64K tokens during pre-training, and from 256K to 1M tokens during the
    cooldown phase.

https://arxiv.org/pdf/2607.24653 page 12

[dannyw]

You absolutely don't need to 'retrain' to reduce your context window. In vLLM it is an inference parameter. Smaller context window, smaller KV, less RAM needed to serve the same volume of requests.

With a lot of architectures, you technically don't need to retrain to extend the context window either; e.g. RoPE scaling; but performance is typically crap.