|
|
|
|
|
|
by bob1029
19 days ago
|
|
|
I had a severe performance issue with streaming messages in a custom chatbot UI. I was able to resolve by enqueuing the streamed token chunks and processing them in batches at requestAnimationFrame boundaries. Forcing an immediate repaint on every received chunk seems to break down in a non-linear way. For small conversations with few messages you almost can't tell the difference. |
|
|