[trollbridge]

This is just a bit exciting, although I wonder how the performance of this will stack up next to the stuff we already do with, e.g., a metal-optimised model which we then load into llama-cpp or whatever. (unsloth is a good example of doing this for you "batteries included").

[ElFitz]

A few months back someone reverse-engineered private ANE APIs and shown some significant performance improvements compared to CoreML and Metal, on both inference and training.

- https://maderix.substack.com/p/inside-the-m4-apple-neural-en...

- https://news.ycombinator.com/item?id=47257931