[loup-vaillant]

Ah, that hard limit. Eurisko did flatten out…

> In this future, the most intelligent agents might be only a few standard deviations above the brightest current humans.

Current methods of doing software are reaching their limits. That doesn't mean we have reached the limit yet. See Squeak, and more recently the Viewpoint Research Institute's work: <http://vpri.org/html/work/ifnct.htm>. When I see Frank (basically a personal computing suite in 20K lines, compilers included), I see a proof that we just do software wrong. The actual limit of what humans can program is probably still far.

Fast intelligence isn't a panacea, but still: imagine an Em thinking 10 times faster than meatware, on a personal computer, capable of copying itself over the network. That alone would be pretty dangerous. Now give it perfect dedication, and enough common sense to avoid most obvious mistakes… Now we could stop it… With another such Em. And then Hanson is back.

[randallsquared]

Eurisko is more legend than history, at this point. As far as I know, the source code was never available to anyone except Lenat, and most of the claims about how effective it was at the beginning were sourced directly from Lenat, as well. The fact that we've never seen anything similarly small and effective (and that Lenat abandoned the entire approach in favor of Cyc) makes me wonder how much of what Eurisko is reported to have done is exaggeration.

Your scenario with the Em that's copiable and ten times faster than a human is exactly what I started this with. :)