[newsbinator]

'Blindsight' was great. Probably in my top-5 Sci-Fi of all time.

I hadn't heard of 'Echopraxia'- thanks for the recommendation!

The main assumption I make with AGI is that we'll be very bad at hitting the sweet spot between 1) "toy", 2)"human-level intelligence", and 3) "beyond human-level intelligence".

I doubt we'll stop at any point between (1) and (2). And then by the time we get even a decimal point beyond (2) and toward (3), we're toast.

To follow your analogy, it would be like birds trying to make a bird-like creature and keep it contained, but overshooting. By the time they realize they've overshot, they've got a human in their cage, and that human has 10,000 bird generations of time to evaluate his environment and think about escaping.

We can guess pretty well that the birds are out of their depth, and were doomed from the start.