[andrewstuart]

People are really freaking out about AI aren’t they?

Why bother? It’s moving super fast, just wait and see what happens.

And even if you could control or regulate it, exactly how would you do that? What would you be regulating/controlling? How would you define it?

And why would you want to anyway? The party has just started, if you think the revolution has arrived, your completely wrong - this is just the beginning - the most amazing stuff is yet to come.

These people begging for the pace to slow, it’s analogous to the newspapers and music companies wanting the internet to slow down as they were being rapidly involuntarily made redundant.

[glenneroo]

Did you read the article or letter? They list a lot of very valid reasons. I won't bother quoting because I would just be copy/pasting the article (which is very short).

For a more detailed list of reasons, go read what AI alignment scientists think, since they have been working on how to align an AI in order to not "turn everything into paperclips" since the 1970s and it seems as though many are rather skeptical about us having any future if we keep up at this rate (spoiler: many believe the end of humanity will occur soon after the creation of any superintelligence): https://www.alignmentforum.org

My personal takeaway: continued development might mean the end of humans and maybe even our planet (if an ASI can deploy nanotech to convert everything into substrate). As it already stands, nobody knows why LLMs work as well as they do - they are already a black box. Sure, plenty of people can explain the math behind each of steps involved: training, matrices, transformers, inference, et al. but it's still a big black box which spits out "magic" answers. You can't just drop a breakpoint inside a model during inference to see what's going on, you will just get a long list of unintelligible floating point numbers at any step of the process.

Your question about regulation is valid... but something needs to be done. I feel like we're standing very close to one of the Big Filters.

Comparing AI/AGI/ASI to anything we have currently seen is probably pointless, they are worlds apart. Would you bother comparing a smartphone to a book? The rate of AI/ML progress is sufficient to measure in hours instead of months or years.

[kromem]

It could also present the only realistic way we avoid the many other human-ending threats we've introduced ourselves.

This isn't a one sided coin flip.

You have nuclear war, emerging bioweapon capabilities, global warming, pollution, and a number of other existential concerns. And yes, AI is one on that list.

But how many solutions to those other items do we realistically have right now? Because it keeps seeming to be that nothing is getting done outside people pointing out the sky is falling.

Quite possibly the ONLY real, well funded, viable solution to these issues would be a straight up deus ex machina being pulled out of our rear ends.

So yeah, it's a scary coin flip that could add to the massive list of other things inevitably going to kill us all, or maybe end up being the coin flip to negate the rest of that list.

So a letter that only points out a possible threat model isn't genuinely pursuing discussion over the ethical considerations - it's just fear mongering. And looking at signatures much further down the list of "concerned parent" or "fearful citizen" it's having the intended (and very sketch) effect.

[kromem]

Also, the letter only presents threats.

Part of the ethical consideration needs to be the opportunity cost a six month delay could cause.

As an example, let's say that GPT-N will cure cancer.

Over a six month period that's at least 5 million people dead if the date it arrives and cures cancers broadly was pushed by that long.

What about negotiating foreign policy treaties to prevent war, or identifying a way to reverse climate changes, or any number of other positive effects?

The fact the letter even positions 'should' such an advanced AI exist as a legitimate question I find pretty gross.

Should we hold back the progress of intelligent life in the universe out of the ego of humanity?

I get superintelligence is an unsettling idea.

But inherent to that name is an indication that there's a seat at the debate table that's currently vacant which might have important and interesting things to say on the subject, and anyone suggesting aborting it is a valid course of action (particularly given the myriad of existential threats we already face from fellow humans) I can only regard as being quite far from super.

[Planktonne]

> It’s moving super fast, just wait and see what happens.

This seems like bad advice in almost any context.

[andrewstuart]

Why?

We didn’t have to slow down and wait when the web arrived…. and tons of people lost their jobs because of the web.

[csomar]

> People are really freaking out about AI aren’t they?

AI touches the domain of software development the most (since we have put a lot of data about it on the Internet). It touches other things too like writing, and design. It doesn't touch things like food delivery, construction, or a farmer.

Currently, they happen to be at the lower tier of society. For some reason. Despite the fact that you can't go two days at a row without food. AI can flip this. There is no need for this army of developers, designers, marketers and bureaucrats. Some people are afraid.

Tl;dr: The people who are freaked out about AI are the people who are bound to lose the most by it.

[Kinrany]

Logistics and farming seem like poor examples: both can be solved with programming and a little hardware.

[DoctorOetker]

> Tl;dr: The people who are freaked out about AI are the people who are bound to lose the most by it.

And some probably used ML assistance to contribute to the draft:

> "At some point, it may be important to get independent review before starting to train future systems, and for the most advanced efforts to agree to limit the rate of growth of compute used for creating new models." We agree. That point is now.

This would lead to emphasis on algorithmic compute efficiency, quickly decreasing reliance on mega-actor cooperation! Hah!