[ISL]

I was part of a research group that extensively trained small neural networks for image-processing in 2001, the high-energy physics community had been using them for many years by that time.

Furthermore, I believe that the PalmPilot's handwriting-recognition engine also had a neural-network component.

Agreed that the usage has increased radically in the last twenty years, but even before the GPU-based revolution, it felt like neural networks were already broadly known and in use across the sciences and engineering. They were just slower :).