[gbighin]

Requirements:

> A decent computer with either a CUDA supported graphics card or M1 processor.

Why so? How does an M1 processor replace CUDA in a way a x86_64 processor can't? Do they use ARM assembly?

[pavlov]

It’s not the ARM core but the integrated GPU in the M1. It has access to the entire main memory unlike a traditional GPU with its own local VRAM.

[gbighin]

Oh, interesting! But does it support CUDA? How is the integrated GPU used for ML tasks?

[hnarayanan]

Both PyTorch and TensorFlow offer backends for Metal that works pretty well on Apple Silicon.

[dagmx]

To add to what people said, most of these ML models target an ML library like TensorFlow or PyTorch.

Those in turn have hardware accelerated backends. Traditionally they’ve only had CUDA backends but Apple ported large chunks of both to Metal as well.

So none of these libraries really target CUDA. In fact they’d run fine without a supported GPU but much slower.

[Filligree]

It does not support CUDA; SD does not require CUDA.

[pavlov]

I believe there’s a Tensorflow acceleration adapter for Apple’s ML API which uses Metal behind the scenes.

[malshe]

pytorch can use the GPUs on M1 macs. Sebastian Raschka's post explains it nicely and shows some benchmarks too. https://sebastianraschka.com/blog/2022/pytorch-m1-gpu.html

From his post:

  if you want to run PyTorch code on the GPU, use torch.device("mps") analogous to torch.device("cuda") on an Nvidia GPU.

[crucialfelix]

In some cases there are operations not supported on mps. For those set:

os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"

and it will run on cpu if some operation isn't supported

[malshe]

Excellent! Thanks