|
|
|
|
|
|
by gpugreg
2 days ago
|
|
|
From the Kimi K3 technical report: Kimi K3 supports a context window of up to 1 million tokens. We achieve this through extending
the context window progressively as training proceeds, following a four-stage curriculum. The
window grows from 8K to 64K tokens during pre-training, and from 256K to 1M tokens during the
cooldown phase.
https://arxiv.org/pdf/2607.24653 page 12 |
|
|