Hacker News new | ask | show | jobs
by d3m0t3p 315 days ago
You can batch only if you have distinct chat in parallel,
1 comments
mlyle 315 days ago
> > if I want to run 20 concurrent processes, assuming I need 1k tokens/second throughput (on each)
link