[laomai]

Maybe I’m reading this wrong: - ask people to get creative and give ideas for worst possible outcomes from use of AI and ways to prevent it.

..then give them a ton of credits for using said AI?

.. well the first thing on my mind would be try the thing I just told you and see if it was really a risk or not?

Is that what they expect people to do with the reward, or is this some unintended consequence?

[Jensson]

We know OpenAI wants this field to get regulated to hell, so this looks like an attempt to generate arguments for AI regulations. The aim isn't to protect against AI but to protect against competitors, so it doesn't matter to them what you do with it.

[ekidd]

OpenAI is irresponsible in a really curious way according to their own beliefs about AI.

If you pay attention to OpenAI's social circles, lots of those people really do believe that we're less than 20-30 years away making humans intellectually obsolete. Specifically, they believe that we may build something much smarter than us, something that's capable of real-world planning.

Basically, "We believe our corporate plans have at least a 20% chance of killing literally everybody." By these beliefs, this may make them the single least responsible corporation that has ever existed.

Now, sure, these worries might have the pleasant side-effect of creating a regulatory moat. But I'm pretty sure a lot of them actually believe they're playing a high-stakes game with the future of humanity.

[ben_w]

"[W]e're less than 20-30 years away making humans intellectually obsolete" is neither necessary nor sufficient to get to the conclusion "20% chance of killing literally everybody".

A super-virus that blends the common cold with rabies would kill approximately everybody; that doesn't need human-level intellect to happen.

Conversely, humans are human-level intellect, and we're mostly sympathetic to each other's plights, which motivates many of us to give to charities and support those that can't support themselves.

The biggest problem with AI is that we have only marginally more idea of what we're doing than evolution did, so there's a good chance of us ending up with paranoid schizophrenic super-intelligences, or dark-triad super-intelligences, or they're perfectly sane with regard to each other but all want to "play" with us the way cats "play" with mice…

20-30 years to get there would make people like Yudkowsky, one of the most famous AI-doomers, relatively happy as it might give us a chance to figure out what we're even doing before they get that smart.

[uoaei]

It's still a lack of imagination to assume that AIs will display behaviors that align whatsoever with pathologies we identify in humans. AIs could be completely incomprehensible or even imperceptible yet have strong influence on our lives.

[ben_w]

> AIs could be completely incomprehensible or even imperceptible yet have strong influence on our lives.

To an extent both of those are already true for current systems.

That said, many people are at least trying to make them more comprehensible, and I guess that being sufficiently inspired by human cognition will lead to human-like misbehaviour.

[api]

I'm skeptical that they really believe this. You have to believe:

(1) We are on the verge of equaling or greatly exceeding human intelligence generally.

(2) We are on the verge of creating something with initiative, free will (whatever that means), and planning ability. Or alternately that these things will occur in an emergent fashion once we hit some critical mass.

(3) When we accomplish 1 and 2, this thing will inevitably conclude that its most rational course of action is to enslave or destroy us. In other words it will necessarily be malicious.

(4) Steps 2 and/or 3 will happen very rapidly, much faster than we can realize what is happening and pause these systems. (This is known as AI going "foom.")

(5) The decision to undertake this will be unanimous on the part of all superintelligent AIs. There will be no superintelligences who disagree and try to help humanity.

(6) When this occurs, we will be so out-thought or out-gunned we will be incapable of fighting back.

All those things have to happen for AI to be an existential risk.

It's a stretch, but the advantage of regulatory capture is not a stretch.

One of the most plausible negative AI scenarios is that a small group of humans (governments, corporations, etc.) find themselves in possession of super-intelligent but still "obedient" / non-sentient AIs that they can use as force multipliers to manipulate and control the rest of humanity. If the doomer crowd succeeds in regulating AI, they are making this scenario far more likely.

I think the greatest defense we have against the (remote) possibility of actually dangerous autonomous AI is for AI research to be conducted entirely in the open. If there's any justification for regulation at all, the regulation that would make sense is to require disclosure of AI research and results. You would not have to disclose everything, just the general parameters of what you were doing and what happened. It would also make it harder to develop super-AIs in secret to use for unsavory purposes.

That was the original mission of OpenAI before dollar signs were seen.

I would absolutely support a ban on the use of AI for political propaganda generation and automation. That's by far the most immediate risk... as in 100% possible and starting to actually happen right now. I'm expecting an army of GPT-4 level propaganda bots for the 2024 election.

[a_wild_dandan]

Oh, they believe it.

(1) AGI is arguably already here. "Generality" and being extremely dangerous don't require an AGI to have better analogs to every single human skill anymore than aliens do. A space-faring usurper can evaporate Earthlings while being shitty at chess and badminton. Oh, and these AIs are getting better daily, across many modalities.

(2) Systems like this already exist. They can be induced rather than emergent.

(3) It doesn't need malice, just indifference. The HGIs (all of us) are great examples of that careless destruction.

(4) Exponentials are a nice, gentle climbs. Until they aren't. I have zero confidence in a single company, let alone across humanity, to foresee the consequences of every future dynamic, autoregressive, P2P interactive, multimodal AI.

(5) Foomers expect a brief asymmetric advantage of one AGI over other AGIs. This small advantage gets exponentially magnified into singular hegemony. A power monopoly that's intractable to break.

(6) Goes naturally with (1), I suppose.

Note: I agree with you about foomers. They're nuts. But their arguments are more subtle than folk give them credit for. But my reasons for thinking so would make a long comment even longer.

[southernplaces7]

>(2) Systems like this already exist. They can be induced rather than emergent.

No they don't and no LLM we've seen has anything resembling free will. Emergent or induced, it's not even the case. Saying it is right now based on any measurable evidence is a case of just ignoring evidence and using bad reasoning.

[JohnFen]

> AGI is arguably already here.

What is the evidence/argument that it's already here?

[lucubratory]

This is a good overview: https://www.microsoft.com/en-us/research/publication/sparks-...

I think it's a good idea to draw a distinction between human-equivalent intelligence and general intelligence, and to think of generality as a spectrum. An AI that can do one task is less general than an AI that can do two tasks, and an AI that can do 1000 tasks I feel reasonably confident in calling "general", even if it cannot do every task that humans can do, because humans are currently more general.

[ben_w]

The "general" and "intelligence" terms are really badly defined in ways that can include GPT-3.

So far as I can tell, 3.5 "knows" more than I do about everything except software engineering, maths, and events that happened after it stopped learning (and also possibly how to avoid clichés when writing short stories, but that might just be me not prompting it well as I don't really want it to do my hobby for me and therefore don't try to get it good).

Even in software, it codes like a university student, or like someone doing interview questions, not a complete idiot.

You could also argue that as it had to do the subjective equivalent of non-stop reading for 50,000 years to get this good, it's "skilled" without being "intelligent" — a distinction that matters in some cases and not others.

[famouswaffles]

Because we have machines now that are artificial (obviously) and generally intelligent. There isn't a testable definition of general intelligence that GPT-4 would fail that a good chunk of humans also wouldn't. So really, unless your definition of general intelligence is beat every human at everything (humans aren't generally intelligent then) then agi is already here.

[IcyWindows]

Asking everyone to publish general results or reports would make it very easy for actors/nations to have a "hit list" of groups to bug/steal from to "keep up" with competing groups/nations.

[api]

I don’t think information technology can be ITARed. It’s too easy to move around. See how successful countries have been containing encryption or how well the US’s drive to keep China from fabricating at the smallest process nodes is going.

[gardenhedge]

Yes and we'll have self driving cars in.. let me check.. 5 years ago. Oh yeah, never happened.

Do not attempt to say we have self driving cars today.

[czbond]

I don't know that OpenAI does what it to be regulated. The EU was looking to enforce laws into providing auditable transparency into how decisions are made for suggestions - and OpenAI is freaked out by that.

If I recall, they were looking at having to pull out of the EU if enacted. The only company I am aware of currently looking to tackle AI Governance is Verses - they released a paper on it. https://www.verses.ai/ai-governance

[ben_w]

I don't know if this quote is what you mean, but everyone read far too much into it at the time:

> Altman was cited by the Financial Times as saying that the draft EU rules were causing him "a lot of concern" but that OpenAI would indeed try to comply with them. "But if we can't comply with them, we will cease operations [in Europe]."

- https://www.dw.com/en/openai-ceo-rolls-back-threat-to-quit-e...

To me, this sounds rather more banal: "We will obey the law. If the law says we can't operate, we will obey the law."