|
|
|
|
|
|
by XRG
503 days ago
|
|
|
If we assume the indicated 400 W load with a total processing time of 20 s per user query, and 20 mio queries per day (‘daily users’), shouldn’t the daily consumption be closer to 44 kWh? (20 mio x 0.4 kW x 20 s / 3600 s/h) Other than that, the energy use for LLMs is obviously substantial. I seem to remember that the operating costs should be fairly evenly split between hardware (i.e., GPU) depreciation and energy costs. |
|
|