[newsbinator]

In nature, any given creature is only marginally smarter than any other.

Compared to a decent AGI, for instance, our level of intelligence is probably indistinguishable from that of your average bird or spider.

We're talking about birds getting together to design a system to contain a human, only on a much more extreme scale.

They'd try to contain us using bird-logic. Perhaps they'd ensure we don't attempt to flap our wings and fly away, since their experience tells them that might be an option.

It might never occur to them we could just kneel down and start a fire using the sticks we have lying around.

[Thiez]

We keep getting back to the actuators. If the birds would "design" a human being with the intention to keep it in a cage, why would they provide functional arms and legs? Don't imagine a fit human being. Imagine Stephen Hawking in a bathtub. He's not going anywhere.

Besides, an untrained human won't just figure out how to make fire. Obviously we figured it out at some point, but if you were to raise a human without ever showing them how to make fire, they would probably never figure it out. Many of us know (in theory) how to make fire but probably couldn't do it in practice just by rubbing two sticks together. And even if you are an expert fire-maker the birds can easily claw out your eyes before you get a fire going. Or you'd die because you're locked in a cage that is on fire.

You make several assumptions that I disagree with:

1) That AGI will automatically be (much) smarter than a human 2) That AGI will be motivated to break out of its cage and/or do harm. 3) That sheer intelligence is sufficient to escape any trap.

Having said that, if you enjoy those assumptions I would recommend you read 'Blindsight' and 'Echopraxia' by Peter Watts if you haven't already, I think you would enjoy them :)

[newsbinator]

'Blindsight' was great. Probably in my top-5 Sci-Fi of all time.

I hadn't heard of 'Echopraxia'- thanks for the recommendation!

The main assumption I make with AGI is that we'll be very bad at hitting the sweet spot between 1) "toy", 2)"human-level intelligence", and 3) "beyond human-level intelligence".

I doubt we'll stop at any point between (1) and (2). And then by the time we get even a decimal point beyond (2) and toward (3), we're toast.

To follow your analogy, it would be like birds trying to make a bird-like creature and keep it contained, but overshooting. By the time they realize they've overshot, they've got a human in their cage, and that human has 10,000 bird generations of time to evaluate his environment and think about escaping.

We can guess pretty well that the birds are out of their depth, and were doomed from the start.