I think companies dream of increasing productivity using AI, so they want to encourage their devs to use AI more. Since it's notoriously very hard to measure software productivity, and they have no way to really measure how well engineers use AI, they measure number of tokens.
Which is dumb. Because a bad engineer might ask the AI to "convert this CSV to JSON" which would burn token, while a good engineer would ask the AI to "write a script to convert this CSV to JSON".
Of course, this example is not ideal, since a SOTA model would probably just write a script anyway in the first case... But you get the idea.
I think companies dream of increasing productivity using AI, so they want to encourage their devs to use AI more. Since it's notoriously very hard to measure software productivity, and they have no way to really measure how well engineers use AI, they measure number of tokens.
Which is dumb. Because a bad engineer might ask the AI to "convert this CSV to JSON" which would burn token, while a good engineer would ask the AI to "write a script to convert this CSV to JSON". Of course, this example is not ideal, since a SOTA model would probably just write a script anyway in the first case... But you get the idea.