[throwaway894345]

I would happily jump ship for any competitor that offers solid AI inference benchmarks at a competitive power efficiency, but as far as I can tell Apple owns that market by a pretty big margin. I’m sure someone will point out if I’m wrong.

[nl]

Ryzen AI are competitive with Macs in absolute and power efficiency terms. There is a Framework 13 with them I believe.

[topaz0]

Humans own inference power efficiency by a much bigger margin

[throwaway894345]

Owning a human is much more problematic, and anyway a human’s peak efficiency might be higher, but humans have to sleep and take breaks so overall probably not.

[rbanffy]

Humans also tend to hallucinate a lot, but it’s not polite to use that word for them. With humans we say they were wrong, because offending a human reduces performance and you can’t reset their contexts.

[pyrophane]

I just have a desktop at home that I run inference off of. It is a great setup and I don't find myself wanting to inference models directly on my laptop.

[throwaway894345]

That’s what I would do too, but I haven’t found a desktop build that can rival a Mac Mini or Mac Studio on performance per watt. I haven’t looked super hard, but it seems like Mac is in a different ballpark.

[zhivota]

I mean, once it's a desktop, watts are pretty cheap, so it's a bit strange to optimize on that factor for the desktop form factor. For laptops it makes a ton of sense.

[throwaway894345]

If I was optimizing solely for cost I wouldn't use a desktop in the first place. Watts are cheap, but they result in pollution and noise (e.g., fan) or you need some costly alternative. Also I just value efficiency.