[sadhorse]

Does every token requires a full model computation?

[onedognight]

No, you can cache some of the work you did when processing the previous tokens. This is one of the key optimization ideas designed into the architecture.