|
|
|
|
|
|
by Freedom5093
294 days ago
|
|
|
I don't understand: > all of the prompt tokens are available in advance and do not require decoding > The other technique is called batching: this technique aggregates multiple prompts into a single decode operation. So do prompts get decoded or not? Are there 2 decode steps? Unclear |
|
|