Hacker News new | ask | show | jobs
by nrhrjrjrjtntbt 183 days ago
Why measure in minutes and not tokens? Seems you could cheat by slowing the ai down.
1 comments

They measure the time it takes a human to complete the task. They don't care how long the AI takes (although in practice it's much faster than human). Measuring tokens isn't a good idea because newer models can complete tasks using fewer tokens.