[whywhywhywhy]

>We can see better performance gains with the m1 when there are fewer weights to train likely due to the superior memory architecture of the M1.

Wasn't this whole "M1 memory" thing decided to be a myth now some more technical people have dissected it?

[gameswithgo]

I think two different memory things are being talked about:

1. There is an idea that M1 has RAM that is vastly higher bandwidth than intel/amd machines. In reality it is the same laptop ddr ram that other machines have, though at a very high clock rate. Not higher than the best intel laptops though. So the bandwidth is not any more amazing than a top end Intel laptop, and latency is no different.

2. But in this case I believe they are talking about the CPU and GPU both being able to freely access the same ram, as compared to a setup where you have a discrete GPU with it's own ram, where data must first be copied to the GPU ram for the GPU to do something with it. In some workloads this can be an inferior approach, in others it can be superior, as the GPU's ram is faster. The M1 model again isn't unique, as its similar to how game consoles work, I believe.

[dragontamer]

> In reality it is the same laptop ddr ram that other machines have

LPDDR4 is more well known for cell phones than laptops actually. I think it shows the stagnation of the laptop market (and DDR4) that LPDDR4 is really catching up (and then some). Or maybe... because cell phones are more widespread these days, cell phones just naturally get the better tech?

On the other hand, M1 is pretty wide. Apple clearly is tackling the memory bottleneck very strongly in its design.

DDR5 is going to be the next major step forward for desktops/laptops.

> 2. But in this case I believe they are talking about the CPU and GPU both being able to freely access the same ram, as compared to a setup where you have a discrete GPU with it's own ram, where data must first be copied to the GPU ram for the GPU to do something with it. In some workloads this can be an inferior approach, in others it can be superior, as the GPU's ram is faster. The M1 model again isn't unique, as its similar to how game consoles work, I believe.

More than just the "same RAM", but probably even shares the same last-level cache. Both AMD's chips and Intel's iGPUs share the cache with its CPU/GPU hybrid architectures.

However: it seems like on-core SIMD units (aka: AVX or ARM NEON / SVE) are even lower latency, since those share L1 cache.

Any situation where you need low latency but SIMD, it makes more sense to use AVX / SVE than even waiting for L3 cache to talk to the iGPU. Any situation where you need massive parallelism, a dedicated 3090 is more useful.

Its going to be tough to figure out a good use of iGPUs: they're being squeezed on the latency front (by things like A64Fx: 512-bit ARM SIMD, as well as AVX512 on the Intel side), and also squeezed by the bandwidth front (by classic GPUs)

[iforgotpassword]

Myth or not, it's memory bandwidth is amazing, so I guess that helps.

[rsynnott]

As with many things, there isn't one "M1 memory" thing. It's a combination of myth and real stuff. No, it isn't ultra-low latency or high-bandwidth. But on the other hand, single core achievable bandwidth is very high.

[coldtea]

No. Some technical people just gave their non-definitive two cents.

[vletal]

Could you please provide some resources on how the unified memory model supposedly works? Why is it a "myth"?

[tyingq]

I believe it's referring not to unified memory, but some speculation that the memory being closer to the CPU makes some notable difference. That line was in a fair amount of the initial articles about the M1, and fits the "myth" description.