[loandbehold]

This is pretty much what underlies AI doomer argument of people like EY. Humans will gradually hand over civilization to black box AI they can't understand. As AI becomes more complex and powerful it will be harder and harder to control.

[pishpash]

That assumes everyone will do so. Some people won't, and it's not clear you need a large number of such, a priesthood if you like, to survive as a species without AI.

[ben_w]

There's a lot of un-contacted (or voluntarily self-isolating, it's hard to tell which) tribes around the world; things go poorly for them when businesses with power tools decide they want the resources of the land they live in, even when they're theoretically protected by law of the nation whose borders both the tribes and the businesses both find themselves within.