Hacker News new | ask | show | jobs
by lzooz 1297 days ago
Does it really generate the text like that, little by little, or is that just a needless animation?
2 comments

It's literally the model's output, in real-time. The GPT-3 API lets you turn streaming on or off, but if you don't enable it you have to wait much longer for the completion to finish entirely, before getting anything back.
Yes because it's the way the AI's architecture works, it generates the next token in a phrase based on the previous tokens. You can see it happen in real time or you can wait until it's all finished.