Hacker News new | ask | show | jobs
by threeseed 1696 days ago
There was this comment on Reddit:

a) ML Compute doesnt (today) use the neural engine. It uses Accelerate.framework's BNNS module and Metal on GPU - meaning its CPU and GPU only.

b) The Apple Neural Engine (ANE) is a private inference accelerator which has no programmable API. Its only accessible via CoreML ML Program or ML Model format, and the CoreML runtime May, or May Not™ decide to run your model on the ANE depending on system load, other tasks, battery / power, and the phase of the moon.

See my post on SO: https://stackoverflow.com/questions/58437789/coreml-mlmodelc...
2 comments
threeseed 1696 days ago
Someone has benchmarked Tensorflow using this code:

https://github.com/apple/tensorflow_macos/issues/25

https://forums.macrumors.com/threads/apple-silicon-deep-lear...

It is expected that the M1 Max should have similar performance to a RTX-2080 or Titan X for that task at least.

link
jjcon 1696 days ago
That is on a extremely small network (from a 1998 paper) running on an extremely small dataset of small images (28px by 28px grayscale images) and they are comparing the ms per step which is going to vary dramatically. Not only that they do not bother to run the m1max and simply say it, "should run twice as fast as the M1 Pro"

In other words that benchmark is completely useless. They should run a standard network from the past decade at least (say VGG16) on useful image sizes and should give the 10 epoch training time if they want anything stable that may approximate hobbyist workloads.

link
dev_tty01 1696 days ago
It has an API, it is called CoreML. The compiler will use whatever resources will run your code the best, which on a phone may consider power consumption as well as performance. I am doubtful as programmers we know more about the hardware performance tradeoffs than the designers of the hardware.

"Core ML optimizes on-device performance by leveraging the CPU, GPU, and Neural Engine while minimizing its memory footprint and power consumption."

https://developer.apple.com/documentation/coreml

Yes, CoreML may or may not run on the Neural engine, depending upon what you are trying to do and the system conditions. It will run your code on whatever works best.

link