[_siis]

This is such an amazingly shortsighted, naive, and flawed way of thinking I'm having a really hard time not sniping.

A large number of people are very concerned about this, and rightfully so because so many people don't get the risks (including yourself it seems).

These people largely aren't fearmongers either, they are experts in that field, serious engineers.

A computer can't do this you say... and that is how it has always been right up until it can, and then whole ecosystems shift seemingly overnight. This will be no different.

Let me ask you, where's the risk management? You deal with anything dangerous you've dealt with risk management. Where is it for this? Can we even evaluate a problem like this? Our main form of interaction is by code, we work in seconds it ticks in nanoseconds, by the time it receives input from us, it could have predicted and nullified our attempts to do anything if it were sentient.

Right now, its very simple, there is almost no risk management, and you and the smartest people in the world trying to tackle this problem are clawing in the dark blind, and you don't know it, but they do and the ones with true intelligence are scared shitless which is why there are so many people going on record (a thing normally that would be a career killer), trying to prevent you from driving everyone over that proverbial cliff, only its more like a dam.

For you and most other people that don't work with this stuff, its an out of context problem that will never happen, and that's fine for small things that don't cascade.

People are traditionally very bad at recognizing cascading failures before it actually happens. This is like a dam with a crack running through it that almost no one has noticed, and your home is right underneath it, in this case everyone's home is underneath it.

What could possibly go wrong with giving someone, really anyone, who doesn't recognize the risks the ability to potentially end everything if the digital dice line up just right.

Literally Everything is networked. Globally.

It doesn't even need to be Battlestar Galactica type apocalypses, though that's fairly realistic pilot about how it might go down if it became sentient. It can also do it without even being sentient by the slow Ayn Rand/John Galt route where societal mechanics do the majority of the work, all you need to do is disrupt the economic cycle between factor and non-factor markets to a sufficient degree, and people will do the rest, plenty of examples where we were able to restart in the historic record, what about those dark areas for which we have no history; without modern technology we can't grow enough food to feed half the worlds current population.

When the stakes are this high, and the risk management is so nonexistent; everyone including policy makers should be scared shitless and do something about it. If you look at things like how the Manhattan project were handled, they were done with more risk management and care for the amount of destruction potential than either bio or cyber.

Our modern day society is largely fully dependent on technology for survival. What happens when that turns against you, or just ceases to function.

[staticman2]

"These people largely aren't fearmongers either, they are experts in that field, serious engineers."

How many "experts in their field" think GPT 4 can end the human race?

[than3]

Currently, 50% of AI researchers think there's a 10% chance or higher that human civilization will be wiped out in the near future as a result of our inability to control AI.

Another person posted a great youtube overview that sums up and covers the broad points in a 1 hour presentation.

I suggest you watch it to catch up. You'll notice one particular thing is absent, AGI sentience doomsday isn't discussed, though it is a valid risk case too, its not what most experts are concerned with. What does concern the experts is the lack of risk management, and the exponential on exponential growth.

With that kind of growth, its not enough to keep pace, you have to predict accurately where it will be and somehow exceed it, two almost impossible problems.

I highly suggest you review the video, and take the time needed to process what the experts are saying before discounting and minimizing something so impactful.

[staticman2]

I'd be interested in what survey you are referring to. What came up in a search is this:

https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/

And 50% of A.I. researchers certainly didn't take that survey.

I also see no reference to the "near future" in the question

The question would include an A.I. destroying the human race 7000 years from now.

But I was mainly responding to your comment that "one might even go so far as to argue a literal public attempt at ending the human race."

Unless GPT-4 specifically is believed to be a threat to the human race your comment was hyperbole.

I'll take a look at that video.

Edited at add: and the video quotes a survey that you were clearly referring to that says nothing about time scale. There's no claim the "near" future is involved.

Also, the question is vague and certainly isn't asking if a Chatbot will destroy the human race.

[than3]

> I'd be interested in what survey you are referring to.

Its the same as mentioned in the video.

> But I was mainly responding to your comment ... > Unless GPT-4 specifically is believed to be a threat to the human race ...

That's flawed logic. A false dichotomy, and also begs the question as to who decides it is a threat.

As for whether its dangerous, I think the fact that the model they discussed in that video shipped and deployed publicly before anyone knew it had embedded knowledge of research grade chemistry capable of some horrific things, all without the knowledge of the people who designed it. It was only discovered after the fact, and that is pretty disturbing.

With dangerous and existential threats, its not considered safe until deemed unsafe, its by-default considered unsafe until deemed its safe. That's how you limit tragedies.

We can disagree, but if we do I sincerely hope you do not touch this stuff.

[staticman2]

It's banally true that intentionally putting an A.I. in charge of our nucleur arsenal might be dangerous.

My point is someone can answer a survey stating A.I. could destroy our species without believing GPT-4 is existentially dangerous.

[than3]

You've changed your argument which causes me to be skeptical of your credibility

Not everyone is equally educated, the two are not mutually exclusive.

People can say either. Educated, rational and reasonable people would say yes on both if they do the risk management analysis and understand the factors driving how it will be used.