Cost per token has cratered a thousand percent over the last two years, and that's not just lighting VC on fire, efficiency gains are made left and right.
How much do we need to progress before it becomes comparable in terms of energy to the (often already rather energy-inefficient) data pipelines we've been using so far?
Recall that while the cost per token may decrease, CoT multiplies the number of tokens by several orders of magnitude.
Recall that while the cost per token may decrease, CoT multiplies the number of tokens by several orders of magnitude.