[EliBullockPapa]

Anyone know a nice iOS app to run these locally?

[simonw]

MLC Chat is a great iPhone app for running models (it's on Android too) and currently ships with Llama 3.2 3B Instruct - not the version Meta released today, its a quantized version of their previous release.

I wouldn't be surprised to see it add the new ones shortly, it's quite actively maintained.

https://apps.apple.com/us/app/mlc-chat/id6448482937

[Havoc]

Seems much more stable than the last time I tried it too

[Arcuru]

I access them by running the models in Ollama (on my own hardware), and then using my app Chaz[1] to access it through my normal Matrix client.

[1] - https://github.com/arcuru/chaz

[drilbo]

https://github.com/a-ghorbani/pocketpal-ai

This was just recently open sourced and is pretty nice. Only issue I've had is very minor UI stuff (on Android, sounds like it runs better on iOS from skimming comments)

[evbogue]

I'm on Android, however my somewhat elaborate solution was to install Ollama on my home laptop computer and then ssh in when I want to query a model. I figured that'd be better for my phone battery. Since my home computer is behind NAT I run yggdrasil on everything so I can access my AI on the go.

[behnamoh]

I've been using PocketGPT.