[randomcatuser]

I find this argument a bit weak:

for example, regarding human marginalization in states, it's just rehashing basic tropes about government (tldr, technology exacerbates the creation of surveillance states)

- "If the creation and interpretation of laws becomes far more complex, it may become much harder for humans to even interact with legislation and the legal system directly"

Well duh. That's why as soon as we notice these things, we pass laws against it. AI isn't posing the "existential risk", the way we set up our systems are. There are lots of dictators, coups, surveillance states today. And yet, there are more places in which society functions decently well.

So overall, I'm more of the opinion that people will adapt and make things better for themselves. All this anthropomorphization of "the state" and "AI" obscures the basic premise, which is we created all this stuff, and we can (and have) modified the system to suit human flourishing

[duvenaud]

> as we notice these things, we pass laws against it

Well, the claim is that that's the sort of thing that will get harder once humans aren't involved in most important decisions.

> which is we created all this stuff, and we can (and have) modified the system to suit human flourishing

Why did we create North Korea? Why did we create WWI? We create horrible traps for ourselves all the time by accident. But so far there has been a floor on how bad things can get (sort of, most of the time), because eventually you run out of people to maintain the horrible system. But soon horrible systems will be more self-sustaining.