[liamcardenas]

I know very little, so perhaps someone could enlighten me. But I am curious how Apple Silicon will be for machine learning.

When Apple releases a MacBook Pro with 64GB of unified memory (assuming they will) — won’t that be amazing for machine learning? I am under the impression that GPU memory is a huge factor in performance. Also, is there any way that the neural engine can accelerate training — or is it just for executing trained models faster?

[_delirium]

I wouldn't expect it to be particularly competitive in training large models. It's an integrated GPU with 8 cores, and the "neural engine" has an additional 16 cores. The kinds of discrete GPUs (mostly Nvidia) that people use for deep learning have more like 5000+ cores.

I think Apple is aiming more at either training small models, or running pre-trained models. For example Photoshop is starting to integrate neural filters, so NN inference performance can be important for some desktop applications.

[tachyonbeam]

I think it's pretty clear they're aiming at inference only. Training models on laptops is never going to be competitive. Might be fun for prototyping small models in PyTorch/TensorFlow though.

[Veedrac]

No, they're aiming at training too.

[aero-glide]

Is it fair to compare cores like that?

[terramex]

It is not. According to [1] M1 GPU can run "up to 25000 threads".

Comparing raw numbers between vendors is always tricky, but it looks like Apple's "cores" are more like Nvidia's 'Streaming Multiprocessors' (SM's), of which their cards have between 14 and 100. M1 seems to perform similar to their older, mid-end desktop cards (1050 Ti has 6 SM's and M1 matches it in benchmarks).

[1] https://www.apple.com/newsroom/2020/11/apple-unleashes-m1/

[sliken]

https://blog.tensorflow.org/2020/11/accelerating-tensorflow-...

So way better than a regular Intel CPU, I've heard comparisons with the GTX 1050. Definitely not going to replace a top of the line GPU for ML.

[m15i]

I too am curious how 64 GB unified memory performs for training deep learning models. Even if speed isn't amazing, 64 GB is much greater than the 24 GB available in Nvidia's flagship consumer cards, which would allow for inputting larger images, bigger batch sizes, deeper networks etc. Also, will be interesting to see how all of the different cores are used.

[tachyonbeam]

It's only got 8GB or 16GB of RAM AFAIK, because the RAM is part of the chip, and more memory would mean a really big die.

[thoughtsimple]

No. This has been misreported. The RAM is on the same package but not part of the same silicon die. Basically the SoC is mounted next to the RAM on a carrier.

[Veedrac]

The true answer is we don't know what Apple's scaled-up GPUs will look like, so it's hard to tell precisely how performant they will be.

However, everything so far indicates these will be pretty powerful, as even the M1 is pretty beastly for what it is. So it's possible.