[rusk]

Not the person asked, but in time honoured tradition I will venture forth that the key difference is billions of years of evolution. Innumerable blooms and culls. And a system that is vertically integrated to its core and self sustaining.

[ben_w]

AI can be, and often are, trained by simulated evolution.

[rusk]

Simulated.

[ben_w]

You have to say why you think that matters. It still culls the unfit.

[rusk]

I don’t. You have boiled a process of billions of years down to a single sentence. You should ponder your absurdity.

[ben_w]

That's the point of language, to abstract a complex thing to what is often as little as a single word or sentence. It's not like the idea represented by the words "simulated evolution" is itself as simple as those two words anyway.

That it takes nature "billions of years" for natural evolution isn't even important here, because it's not like simulations have to run in real-time.

If you run simulated evolution with mechanical parts and the reward function of things that function like clocks, you get the (design of) a thing that functions like a clock, and if you run the physics simulation of the design, you can tell the time with it. Do it with electronics and things that act like a radio, you get a radio. Do it with a CAD design and the goal of strength for minimum mass, you end up with something that looks bone-like.

We also do it with AI, why should we expect it not to produce things in the general category of "minds"? Not necessarily human minds, even the biggest by parameter count are much smaller structures than our brains, but the general category.