[sponaugle]

What I was commenting on was the concept that a small model at home is somehow more efficient. To make a reasonable and fair comparison you would compare many people running a small model at home vs those same people using what would likely be a shared resource in a datacenter.

The core concept is that tokens/watt is tokens/watt ( for a given model of course ). A computer at home is actually less efficient overall because most of the time it is not doing tokens but still using a small footprint of power.

The revenue pressure is an interesting problem , but I suspect the actual demand math will be much more complicated.

I find local models interesting for sure, and run several on my own personal DGX cluster. I am however most certainly not power efficient!