[theahura]

I think it's incorrect that the insight has gone nowhere. See, for eg, contrastive loss / clip, or vqgan image generation. Arguably also diffusion models.

More generally, in my experience as an AI researcher, understandings of the geometry of data leads directly to changes in model architecture. Though people disparage that as "trial and error" it is far more directed than people on the outside give credit for.

[Graviscalar]

The geometric intuition is solid, but actually applying topology has been less fruitful in spite of a lot of people trying their best, as Chris Olah himself has said elsewhere in this thread.