[PeterisP]

Wait some years until mobile hardware is sufficiently powerful to run similar models locally.

Or don't do this at all, and rely on cloud models like most other solutions - running things locally has some benefits for privacy, control and perhaps cost, but you can get all of the same functionality without a local model.