[blobbers]

Have you tried optimizing for MLX? It seems like a waste to have neural cores and not use them.

I've often wondered why the hype around apple neural core when 99% of software doesn't use them.

[genxy]

Yeah, first think I looked for on the post was MLX and it wasn't there.

https://github.com/ml-explore/mlx-lm

Having used half the systems that Vicki mentioned, mlx was the best balance between power and ease of use. Just a pip install away.