Y
Hacker News
new
|
ask
|
show
|
jobs
by
SoulAuctioneer
827 days ago
It's more about UX, to reduce the perceived delay. LLMs inherently stream their responses, but if you wait until the LLM has finished inference, the user is sitting around twiddling their thumbs.