|
|
|
|
|
|
by matt-p
300 days ago
|
|
|
188M input / 80M output tokens per hour was per node I thought? Reversing out these numbers tells us that they're paying about $2/H100/Hour (or $16/hour for a 8xH100 node). Disclaimer (one of my sites) https://www.serversearcher.com/servers/gpu - says that a one month commit on a 8XH100 node goes for $12.91/hour. The "I'm buying the servers and putting them in COLO rate" usually works out at around $10/Hour, so there's scope here to reduce the cost by ~30% just by doing better/more committed purchasing. |
|
|