[rcarmo]

You just don’t have the bandwidth to do that. Even if you use the M.2 slot you’ll be significantly bottlenecked and would be better off using something else - even an AMD iGPU will work better (https://taoofmac.com/space/blog/2024/04/13/2100)

[c0g]

What’s the bottleneck? Once I’ve got the model and data onto the GPU my only cost is launching CUDA kernels right?

Not sure if that blog post is relevant, but even if it is it shows a 3060 gets //way// faster throughout than the igpu it is testing. I suppose I can test this myself by plugging my 3070 into the NVME on my desktop.