[Animats]

How to organize them is a hard problem. For general-purpose use, we have only three architectures today - shared memory multiprocessors, GPUs, and clusters. When Hopper gave that talk, people were proposing all sorts of non-shared memory multiprocessor setups. The ILLIAC IV and the BBN Butterfly predate that talk, while the NCube, the Transputer, and the Connection Machine followed it by a year or two. This was a hot topic at the time.

All of those were duds. Other than the PS3 Cell, also a dud, none of those architectures were built in quantity. They're really hard to program. You have to organize your program around the data transfer between neighbor units. It really works only for programs that have a spatial structure, such as finite element analysis, weather prediction, or fluid dynamics calculations for nuclear weapons design. Those were a big part of government computing when Hopper was active. They aren't a big part of computing today.

It's interesting that GPUs became generally useful beyond graphics. But that's another story.

[mikaraento]

(Some) TPUs look more like those non-shared memory systems. The TPU has compute tiles with local memory and the program needs to deal with data transfer. However, the heavy lifting is left to the compiler, rather than the programmer.

Some TPUs are also structured around fixed dataflow (systolic arrays for matrix multiplication).

[sillywalk]

> non-shared memory multiprocessor setups

It's not really comparable to the other examples you cite - the ncube/transputer/connection machine, in that it was programmed conventionally, not requiring a special parallel language, but Tandem's NonStop was this, starting in ~1976 or 1977. Loosely coupled, shared-nothing processors, communicating with messages over a pair of high-speed inter-processor busses. It was certainly a niche product, but not a dud. It's still around having been ported from a proprietary stack-machine to MIPS to Itanium to X86.

EDIT: I suppose it can be compared to a Single System Image cluster.