[ximeng]

ChatGPT estimates a user that runs all the LLM widgets on this page will cost around a cent. If this hits 10,000 page view that starts to get pricy. Similarly for running this at Google scale, the cost per LLM api call will definitely add up.

[pmarreck]

Locally-running LLM's might be good enough to do a decent enough job at this point... or soon will be.

[nthingtohide]

One more line of thinking is : Should each product have an mini AIs which tries to capture my essence useful only for that tool or product?

Or should there be an mega AI which will be my clone and can handle all these disparate scenarios in a unified manner?

Which approach will win ?

[Kiro]

They are not necessarily cheaper. The commercial models are heavily subsidized to a point where they match your electricity cost for running it locally.

[pmarreck]

In the arguably-unique case of Apple Silicon, I'm not sure about that. The SoC-integrated GPU and unified RAM ends up being extremely good for running LLM's locally and at low energy cost.

Of course, there's the upfront cost of Apple hardware... and the lack of server hardware per se... and Apple's seeming jekyll/hyde treatment of any use-case of their GPU's that doesn't involve their own direct business...

[recursive]

The energy in my phone's battery is worth more to me than the grid spot-price of electricity.