[_delirium]

I wouldn't expect it to be particularly competitive in training large models. It's an integrated GPU with 8 cores, and the "neural engine" has an additional 16 cores. The kinds of discrete GPUs (mostly Nvidia) that people use for deep learning have more like 5000+ cores.

I think Apple is aiming more at either training small models, or running pre-trained models. For example Photoshop is starting to integrate neural filters, so NN inference performance can be important for some desktop applications.

[tachyonbeam]

I think it's pretty clear they're aiming at inference only. Training models on laptops is never going to be competitive. Might be fun for prototyping small models in PyTorch/TensorFlow though.

[Veedrac]

No, they're aiming at training too.

[aero-glide]

Is it fair to compare cores like that?

[terramex]

It is not. According to [1] M1 GPU can run "up to 25000 threads".

Comparing raw numbers between vendors is always tricky, but it looks like Apple's "cores" are more like Nvidia's 'Streaming Multiprocessors' (SM's), of which their cards have between 14 and 100. M1 seems to perform similar to their older, mid-end desktop cards (1050 Ti has 6 SM's and M1 matches it in benchmarks).

[1] https://www.apple.com/newsroom/2020/11/apple-unleashes-m1/