[ada1981]

Do you have any insight / resources into developing a local AI?

I have a novel use case as a wilderness first responder and am looking for a pathway.

[shuhongwu]

1. llama.cpp can run on both Android and iOS devices. 2. For non-transformer models, Mamba and RWKV are also good options. 3. Additionally, you should have a better understanding of embeddings, tokens, and the structure of transformers.

And on Apple Silicon chips, MLX is preferred and is often the best choice.

[ada1981]

Thanks!

I’m trying to create something that will run online when connected, run local when offline, and dynamically download relevant files to prepare for offline use.