Hacker News new | ask | show | jobs
by modeless 927 days ago
Are you using batch size 1 with LLMs? Larger batch sizes get much higher utilization.