[Footpost]

   Co-evolving parasites

This paper strikes me as the earliest manifestation of what would later become GANs (generative adversarial networks). Yes, the mechanism is different (GAs vs NNs), but the spirit (having a competitive mechanism for speeding up local search) is similar.

[cr0sh]

One individual that has always interested me - and for some reason, has seemed to be "black-holed" by those in computer science (but maybe that is my own perception of things) is Hugo de Garis:

https://en.wikipedia.org/wiki/Hugo_de_Garis

A relatively eccentric person, in the late 1990s - early 2000s he along with others created a hardware system called the CAM Brain Machine (CBM) - his bio above contains more information about it. Numerous papers regarding it were published, some of which can still be found on the internet today.

The machine itself never achieved the successes he marketed it as being able to achieve, but it did seem like an interesting project. Much like Babbage, though, de Garis was constantly moving to "sell" the system to get more government grants to do more research with the machine, hopping from one place to another as the grants (or the government's patience) ran out and cancelled the project.

Perhaps that's why you don't here much about him any longer:

A combination of his eccentricity, his work not panning out as expected (indeed, from what I understand, the whole idea of "evolving neural networks" didn't pan out), his funding model likely sowing seeds of discontent among potential funding sources (that future AI/ML researchers would run up against, generating animosity or disdain toward him?), plus his later ideas being a bit too far out there for researchers to continue to take him seriously (expressed in his work of fiction "The Artilect War"); all of that contributing to his imposed seclusion from the subject matter...

While the idea of creating a "robotic kitten" (Robonoko) controlled by an "evolved neural network" didn't work out, the machines he created are still among the most "aesthetically pleasing to look at" computers around (I place them second only to the Cray Supercomputer designs); they were systems that weren't only technically advanced for their purpose, but also pleasing to look at and display (handy from a marketing perspective I suppose). Only a handful of them were built - I often wonder what happened to those machines; at least one example should belong in a museum - ideally CHM in Mountain View.

[p1esk]

the whole idea of "evolving neural networks" didn't pan out

To the contrary, evolving neural networks is all the rage these days ("Neural Architecture Search", e.g. SOTA on Imagenet [1])

[1] https://arxiv.org/abs/1905.11946

[alexmlamb]

It's interesting that some of his intuitions seem really really good like the idea that you'll get better results by using very large models. On the other hand, specialized hardware before the software works well has always been a disaster because it's so hard to iterate on new hardware. His idea of using evolution as a primary learning process also doesn't work well.

[ericol]

Social networks are a marvelous thing [1] (Yeah, somebody "stole" the idea in your comment)

[1] https://old.reddit.com/r/MachineLearning/comments/d05nfr/d_i...

[Footpost]

That's interesting. I certainly did not make the Reddit post. Since there is interest in proto-GANs, here are two more.

- J. Schmidhuber, Learning Factorial Codes By Predictability Minimization. (1992)

- W. Li, M. Gauci, R. Gross, A Coevolutionary Approach to Learn Animal Behavior Through Controlled Interaction. (2013)

Schmidhuber's work will be widely known (and its relationship with GANs controversial), but I can't recall where I read about Li et al.

[p1esk]

I stole your idea :) Imagine how Danny Hillis must have felt when he heard about Goodfellow's paper!