| This is more likely the junior camper version of "not everything that counts can be counted, and not everything that can be counted counts." In the early days of LLMs, we saw the classic hype-driven bi-modality of opinions. Folks were in the "fake news, fad" camp, or they were in the "omg, take over the world" camp. Those of us closer to the space, with the awareness to know that there was some truth (and a lot of misjudgment) to go around, were in the middle of nowhere. When I co-wrote some driver code with Chat GPT, other engineers (and even one of our directors) told me to keep it quiet. At the same time I had directors and VPs asking me how we could accelerate adoption. For a while, I had access to a cheat code just because I had the audacity to not ask for permission. Folks were sure I would get in trouble for spending thousands per month in LLM operation, but a handful came along for the ride, burning tokens like firewood and learning along the way. Tokenmaxxing is probably coming from at least a few things: 1. A course-correction for the practiced frugality that kept folks from jumping in and just learning at the ragged edge. 2. A willful and deliberate recognition that the best innovations in the later phases of a disruptive introduction often come from sparks of ideation in concentrations of activity. In other words, we don't know where good is, and we need to find it. (Charitable interpretation from the article) 3. Recognition that, even if they don't know why, leaders and product owners will get punished for not jumping in and, because of bullets 1 and 2, won't get punished for trying and missing. Even if they have no idea what they're doing, they're going to fake it until they make it (or slide into another job). This last set is where the pain lives. An organization with healthy and increasing AI tool
usage will see elevated token counts, but so too will one using LLMs to rewrite wikipedia articles without the letter "m" to keep token counts high. These are pathological behaviors brought on by conflated metrics. We had discussions about this in the early LLM days, where my old team was looking to ship new capabilities for older products. There was a lengthy VP-level discussion about getting to "80% usage" of the new system vs the old. Because the new system was a superset of the old, I eventually said "we can do that immediately, but it's a cost goal, where we're just aiming to make our business more expensive to operate, rather than a value goal for our users". We didn't adopt the target, but folks were understandably frustrated that they didn't have a straightforward way to measure and report progress. Tokenmaxxing is, inevitably, a conflated goal, but it's what we have right now. Take advantage of the moment, learn, build, and keep an eye on levers for efficiency. |