Hacker News new | ask | show | jobs
by SoulAuctioneer 827 days ago
It's more about UX, to reduce the perceived delay. LLMs inherently stream their responses, but if you wait until the LLM has finished inference, the user is sitting around twiddling their thumbs.