[amosjyng]

How are you collecting your metrics on token usage and reliability?

[vidarh]

They are from my own runs, with reliability measured in terms of passing extensive test suites. So caveat is that this applies for my specific use and might well vary greatly.