[whoistraitor]

What I’d love to see is a rundown of new efficiencies in generative AI. With things like quantization and specialized transformer hardware, the costs will hopefully be less shocking in the future. And fwiw, I don’t find the article’s stats very shocking anyway. At least it’s a net win type of game, whereas the previous environmental bugbear, PoW crypto, was entirely premised on net-energy loss.

[fooblaster]

All optimizations inevitably end up just being funneled into larger and larger models. The issue with AI is its seemingly endless ability to scale in size for marginal gains in model performance (log(n) scaling essentially).

[GaggiX]

>All optimizations inevitably end up just being funneled into larger and larger models.

Well of course if you're trying to beat the SOTA, bigger sizes allow for better models, but not everyone is trying to use or train the latest SOTA model, maybe llama-3 8b is perfect for what you need to do and having better optimizations to run it locally is gold.

[fooblaster]

Defining "good enough" is something that rarely seems to happen.