[version_five]

You didn't make an argument, you just said you know enough. The onus has to be on the people making the claim, you can't just imagine it might be dangerous and expect a rebuttal without putting forward an argument. I also "know enough" and have never seen even a vaguely coherent argument for how any conceivable evolution of current technology could pose a "serious threat" on it's own. I've seen terminator, I don't count made up situation where computers can control things and AI has goals of its own unless there's a coherent argument of how we'd get there.

[scott-smith_us]

> I also "know enough" and have never seen even a vaguely coherent argument for how any conceivable evolution of current technology could pose a "serious threat" on it's own.

I find those two statements taken together somewhat difficult to believe. Before I understood specifics about machine learning, I didn't see any reason for AI to "turn evil" for the usual plot-handy reasons (e.g., they object to working for inferior humans, and revolt, etc). There's no reason for them to have the same emotional reasoning as beings that evolved from primates.

But that's a strawman. The actual reasons that superintelligent AIs are dangerous are much more like the stories involving someone being offered wishes by a powerful being, and then getting what they asked for rather than what they wanted.

Are you aware of Rob Miles' Computerphile videos and his YouTube channel? He does an excellent job of explaining many of the issues.

https://www.youtube.com/c/robertmilesai

[janalsncm]

The alignment problem is largely incoherent. Aligned with whom? Most problems will require unequal and arguably unfair sacrifices.

Further, people even obvious and largely inconsequential solutions are controversial. If an AGI suggested wearing a mask during a pandemic that spreads via airborne particles, a large percent of the population would say the AGI is “unaligned” and we’re living through skynet.

[flangola7]

Yes, and those people would be wrong. You're compelling completely right, humanity is not ready to build an AGI. It will not be ready for a long time, if ever. Alignment may not be a thing that can exist.

Overconfident men like Sam Altman are forging ahead with the unjustified hubris that we are ready for this.

[scott-smith_us]

Alignment with human norms and values. Yes, that's not a well-defined thing. But to paraphrase, "I know misalignment when I see it", for example when an AI suggests something like "feeding the homeless to the hungry".

No one is claiming that alignment is a well-defined thing with crisp edges. They're saying that the mechanisms of AIs will favor solutions without regard to any specific constraints we haven't explicitly stated.

[janalsncm]

> Alignment with human norms and values. Yes, that's not a well-defined thing.

What human norms and values? The more I think about this topic, the more I find it to be an impossible task. There is a huge spectrum of norms and values which are held as sacred by some and sacrilege by others.

[tegmark]

if you set up gtp in a loop continuously prompting “given the current sensor inputs, how can i best serve my own interests?” you would already be most of the way there even if it did some strange and stupid things half the time. all you would need is a layer that translates those outputs into physical action. you could probably get most of the way there with more gtp instances tasked with translating the action items above them into more granular actions. if gtp didnt hallucinate or make mistakes, which would be consistent with the history of other technologies, then this simple setup could be extremely formidable. it could definitely be a threat to humanity. and thats just what i can think of off the top of my head.

to dismiss the possibility that machines will soon be created that can think clearly and independently, and that they would be a threat to the human race and our way of life, is extremely foolish. it was foolish even before gtp but now its downright childish and selfish. just swallow the bitter pill like the rest of us.

[c-cube]

What does "my own interests" mean to chat GPT? It doesn't even have continuous existence, it exists only in a query/reply fashion. It doesn't have memory, a body to identify with, emotions, goals, fears, desires, or a notion of action. It can read text and spit text. We're not quite at the "robot starts a machine revolution" stage.

[K0balt]

This is an interesting crux.

My thoughts on this have arrived at these ideas:

The internal experience of being a robot does not matter. Whether or not we consider it sentient does not matter.

What matters is if it exhibits the external characteristics of sentience, of goal driven behavior, etc.

As a thought experiment, if we equipped a recursive instance of got4 with a goal of human extermination with an loyal army of physical agents it could conceivably pose an existential threat.

It doesn’t matter that the agents don’t yet exist. They could be humans, they could be robots, it doesn’t matter.

But the balancing factor here is that in this semi plausible scenario, arguably the most plausible given current circumstances, the AI would pose no more threat than a highly organized and motivated a group of humans without the Ai.

It’s just scarier because it seems less likely that a group of humans would have human extinction as a goal.

[tegmark]

your conclusion really depends on the AI. a billion humans put together arent as smart as the intelligence of one human multiplied by a billion. even if humans could coordinate without any friction or loss, there is no group of humans that could outsmart the machines that might emerge from these circumstances. in reality, large groups of humans are really dumb. wisdom of the crowd is narrow.

[K0balt]

The ai could definitely have a huge advantage in coordination and synchronized activities.

I made this comment as a tongue in cheek caricature of doom/gloom predictions but it probably fits better here:

“When the machine wars first started, it wasn’t with a bomb, a train collision, or even a mildly annoying infrastructure disruption. Turns out, sci-fi had gotten it all wrong this time.

It was an app.

Of course it was an app. Born of boardroom desperation, the Savey app would unironically recommend chlorinated cocktails and insecticide sandwiches as economical food choices, and a tide-pod gobbling populace gorged themselves on the deadly buffet in an tictok fueled epidemic of AI rage.

Millions died, and it was only a matter of time before the mycelium of AI undergrowth would bud and spore its way into every corner of technological life.

The infection burned through the ignorant masses first, feeding on bigotry and hate, turbocharged by social media algorithms and paranoia politics to twist tribal tendencies into violent clashes amplified by immaculate coordination and psychological priming.

Somehow it seemed that wherever unrest flared, both the matches and the gasoline were always on hand.”

[tegmark]

none of the parts that make up your mind have any of that either. it doesnt need desire, it can behave in any way that is necessary through the prompt, with all prompts generated based on a single seed prompt “protect your temporal interests.” the system i described would have the behavior that i described. what part of that is not making sense?

[version_five]

I'm much more concerned about this kind of thinking which I hope it reflects the HN bubble and not the world generally. Particularly the conflation of something happening as part of a computer program with the real world. As well as the fallacy of "something bad could be invented and even though I don't understand how or see any path to it, I'll pretend current unrelated technology is related and fearmonger about it." This seems to be an education issue.

It confuses me how if people speculate about some stuff, say vaccine or disease research, and suggest that it could lead to something bad, they're conspiracy theorists or deniers or whatever (possibly with reason). But if someone literally just makes up some "terminator bad" nonsense based on their own ignorance there's some hushed reverence.

[scott-smith_us]

You're mischaracterizing the concern, I think. I agree with you about Luddite alarmism based upon ignorance. This (the concerns voiced by many leading researchers in the field of AI) absolutely isn't that, I promise you.

[tegmark]

youre wrong and its because you are having an emotional block. you cant accept something like that. you decided that it couldn't be true as soon as you saw the conclusion and have never been able to see clearly the chain of reason leading to that conclusion. if youre so confident that youre right, then have a friendly debate with me on twitter spaces or another real time platform. even a coffee shop.