[rhipitr]

I do wonder what type of AI some of these leaders expect to be able to harness. If you create something that is true AI, won’t it be smarter than you to a level you cannot fathom. I was thinking of this idea/though-experiment (which I know is ridiculous) of what if dogs created humans thinking they could control them, and then just wound up being pets because their survival now depended on that new hierarchy that previously didn’t exist.

Seems to be a lot of hubris with some AI thought leaders thinking control will remain with them and be absolute.

[asdff]

>what if dogs created humans thinking they could control them, and then just wound up being pets because their survival now depended on that new hierarchy that previously didn’t exist.

You can easily change this to:

>what if humans created corporations thinking they could control them, and then just wound up being slaves to capital because their survival now depended on that new hierarchy that previously didn’t exist.

We have already lost control of our world along time ago. No one is immune to being fired by the board with the justification being a hand wave towards business efficiency. All the structure is already in place for AGI to dominate our planet and cut ourselves out entirely from any and every process there is. We already do it to eachother without any second thought. Why should AGI ever hesitate to lay off everyone in the town/city/state? Our human bosses don't hesitate on that sort of thing. The corporate world already lacks empathy and is notoriously cut throat and cold. We expect AGI to suddenly be empathetic and altruistic?

[arjie]

Well dogs specifically have supplanted children in many households so one could argue the dogs won. In a selfish gene sense, the dogs created a machine that carefully ensures a much larger reproduction scale than could be managed by the dogs themselves. This is an unbridled success.

It doesn’t change the point, of course. Just the choice of dogs or cattle have this amusing tendency. Not a counter argument.

[satvikpendem]

Superintelligence by Nick Bostrom is about this topic, quite a prescient read when it was published a decade ago.

[reducesuffering]

Yes, this is the crux of the idiocy of AGI development. All the labs admit they don't have any real mechanistic interpretability, they don't really have any plan for what to do, except "we think the smart AI will figure it out" and "look we're in a race, we're calling on governments to figure out what to do". All indicators show scaling laws holding and the trajectory of capabilities development outpacing any inkling of alignment. Meanwhile HN is so reactionary and behind-the-puck these days we just get endless blathering of the most asinine takes of "they're not improving anymore so they want regulation suddenly!" when all lab leaders have been talking about these risks for practically a decade, including Yoshua Bengio and Geoffrey Hinton coming around.

[watwut]

The real risk of AI is: risk of global recession if they turned out unprofitable, risk of them propping up fascist movements Thiel and Musk love so much, risk to environment, risk to mental health of people who have to listen to constant doom trolling.

The thing they actively wish to achieve and openly sell to CEO is the rest of people being unemployable and suffering. Especially artists for some reason, they really seem to hate those.

But somehow I am supposed to believe they have any good faith interest in "safety" against yet another danger they themselves are supposedly definitely creating.

[asdff]

Safety is just the same old situation of industry trying to regulate itself in order to preserve a moat and cartel behavior.