An LLM in my pocket is truly a mind-blowing concept, I have to say. More than anything else — phone, camera, internet. The feels like a really big deal.
And with regard to LLMs (AI?) in general, I don't think right now we have any idea what we will all be using them for in ten years. But it just feels like a fundamental change is coming from all this.
I’ve tried their 1.1B model. The only hiccup was that it seems to require mlx 0.10.0 which is what’s in requirements.txt. You also have to place the llama tokenizer file into the model dir - they do not distribute it. The models published for MLX do not seem to be instruction tuned, so with their default prompt they get repetitive. But I suppose you could convert the instruction tuned checkpoints with the script in the repo.
And with regard to LLMs (AI?) in general, I don't think right now we have any idea what we will all be using them for in ten years. But it just feels like a fundamental change is coming from all this.