This already exists: https://www.cerebras.ai/chip
They claim 44 GB of SRAM at 21 PB/s.
Waferscale severely limits bandwidth once you go beyond SRAM, because with far less chip perimeter per unit area there is less place to hook up IO.
Waferscale severely limits bandwidth once you go beyond SRAM, because with far less chip perimeter per unit area there is less place to hook up IO.