Hacker News new | ask | show | jobs
by r0b05 34 days ago
It would be more interesting to know how many simultaneous users this setup can serve. Otherwise I can just say it serves 500 users but not all of them use it at the same time which doesn't communicate the right level of detail.
1 comments

Depends on TTFT and tokens per second you want.