[deepGem]

Any idea what the sRAM to uRAM ratio is on these new GPUs ? If they have meaningfully higher sRAM than the Hopper GPUs, it could lead to meaningful speedups in large model training.

If they didn't increase the memory bandwidth, then 512GB will enable longer context lengths and that's about it right? No speedups

For any speedups You may need some new variant of FlashAttention3 or something along similar lines to be purpose built for Apple GPUs.

[astrange]

I don't know what you mean by s and u, but there is only one kind of memory in the machine, that's what unified memory means.

[saagarjha]

I assume they mean SRAM versus unified (D)RAM?