a) ML Compute doesnt (today) use the neural engine. It uses Accelerate.framework's BNNS module and Metal on GPU - meaning its CPU and GPU only.
b) The Apple Neural Engine (ANE) is a private inference accelerator which has no programmable API. Its only accessible via CoreML ML Program or ML Model format, and the CoreML runtime May, or May Not™ decide to run your model on the ANE depending on system load, other tasks, battery / power, and the phase of the moon.
That is on a extremely small network (from a 1998 paper) running on an extremely small dataset of small images (28px by 28px grayscale images) and they are comparing the ms per step which is going to vary dramatically. Not only that they do not bother to run the m1max and simply say it, "should run twice as fast as the M1 Pro"
In other words that benchmark is completely useless. They should run a standard network from the past decade at least (say VGG16) on useful image sizes and should give the 10 epoch training time if they want anything stable that may approximate hobbyist workloads.
It has an API, it is called CoreML. The compiler will use whatever resources will run your code the best, which on a phone may consider power consumption as well as performance. I am doubtful as programmers we know more about the hardware performance tradeoffs than the designers of the hardware.
"Core ML optimizes on-device performance by leveraging the CPU, GPU, and Neural Engine while minimizing its memory footprint and power consumption."
Yes, CoreML may or may not run on the Neural engine, depending upon what you are trying to do and the system conditions. It will run your code on whatever works best.
a) ML Compute doesnt (today) use the neural engine. It uses Accelerate.framework's BNNS module and Metal on GPU - meaning its CPU and GPU only.
b) The Apple Neural Engine (ANE) is a private inference accelerator which has no programmable API. Its only accessible via CoreML ML Program or ML Model format, and the CoreML runtime May, or May Not™ decide to run your model on the ANE depending on system load, other tasks, battery / power, and the phase of the moon.
See my post on SO: https://stackoverflow.com/questions/58437789/coreml-mlmodelc...