[simondotau]

Presumably at some point the rapid progress of models will plateau, at least insofar as a model could be frozen in time and remain economically useful for the expected life of hardware. Especially if it comes with compelling benefits e.g. dramatically lower latency and/or dramatically higher performance per watt.

If you can build chips that could run one specific LLM 100x faster than anything else, it would have a use case that nothing else could match.

[lsaferite]

Those taalus chips apparently run at 1/10 the power as the current SOTA GPU setups. If they can execute even partially on their plan, it'll be a literal game changer.

[fragmede]

https://www.cerebras.ai/ is exactly that! Holy shit it's fast.

[otterley]

Cerebras is not that. Cerebras isn’t tied to a particular model like Taalas is. The latter is even faster than Cerebras.

[jwHollister]

Yeah at a certain point if inference is fast and cheap, do I care if it's the smartest? I can iterate in a tighter loop.