[statuslover9000]

> The expert believes that “asking for regulations because of fear of superhuman intelligence is like asking for regulation of transatlantic flights at near the speed of sound in 1925.”

This assessment of the timeline is quite telling. If supersonic flight posed an existential threat to humanity, we certainly should have been thinking about how to mitigate it in 1925.

[fulafel]

1925 of course would have been a great time to put limits on fossil fuel use in aviation along with the rest of the fossil fuel applications to manage the biggest current threat to human civilization. (Arrhenius did the science showing global warming in 1896 or so)

[spacebanana7]

Given the dual use of fossil fuels between military and civilian purposes, I wonder whether any state that deliberately handicapped car/aero/petrochemicals would’ve been able to survive the early twentieth century.

Both the USA and Nazi Germany benefited massively from have a civilian industrial base that was complementary to military production.

[fulafel]

Of course you could also argue that Germany wouldn't have had the early successes in war, (if they had even started it). Or at a third juncture, would have fared worse against USSR.

[oceanplexian]

There's a book called Freedom's Forge that I'm a fan of, it makes the argument that the Auto Industry (And assembly lines, mechanization in general) were the single most important reason the Allies won WWII. In fact all the big auto manufacturers of the time retooled their assembly lines to build tanks and airplanes. It's conceivable that if we never mass produced cars, the US wouldn't have had the capability to win the war.

[echelon]

Miami is still above water.

Would you shut down the powerhouse of our economy -- travel, transportation, energy -- for something hypothetical that hasn't even happened and doesn't appear to be close to happening?

I'm pro-clean energy, but you can't do without fossil fuels. Not if you want society to keep climbing up and up and up.

[lern_too_spel]

Before you jump to policy making, you should get the implications right.

"The current rate of sea level rise at Pensacola Bay has accelerated rapidly since 2010."

"The difference in sea level rise over the last 100 years has been approximately 10 inches—but in the next 75-100 years, the increase in sea level rise could be close to 48 inches." https://blogs.ifas.ufl.edu/escambiaco/2023/04/12/weekly-what...

[CamperBob2]

That sounds like a great way to lose an upcoming world war to some people who DGAF about pollution, climate, or other people in general.

[Simon321]

Well it could be argued that it does, what about supersonic nuclear missiles?

[oh_sigh]

But AI doesn't pose an existential threat to humanity, so we're all good.

[HKH2]

I really can't grasp how people think that a system that doesn't have a need to preserve itself will somehow start thinking for itself.

AI is quite troublesome for privacy though. How much privacy humans need is a question we'll probably have answered the hard way.

[HeatrayEnjoyer]

Who said anything about thinking for itself?

A thing does not require intent or consciousness to be dangerous. How many chemists have blown themselves up because they didn't realize an experiment was dangerous? How many production systems have crashed because the developer didn't accurately predict what the code they wrote will do?

Alkali metals and C++ code do not require ill intent, but they will still obliterate your limbs / revenue if you build and use them wrong.

One of my more tangible hypotheses is a sort of runaway effect. Economic, geopolitical, and military competitive pressures will quickly push out anyone and anything that still relies on last era human-in-the-loop processes, the same way any organization that doesn't utilize artificial lighting, electricity, and instant communication will obviously be left far behind. You have to just trust that the machine running stock market transactions will do its math right.

But unlike transaction software failure modes, which quickly result in outright crashes or verifiably incorrect errors, failure modes of non-bayesian decision making software probably looks something like what happens when existing economic, geopolitical, and military decision-makers make decisions that are harmful, unethical, or otherwise undesirable for humanity. This time augmented with, if not superhuman intelligence, at least superhuman speed and superhuman knowledge breadth.

[sumanthvepa]

Love that observation on C++. That's the reason I love C++. It's a language for those who need, nay crave, absolute raw performance. No training wheels. Short of assembly, it's just as close to the machine as you can get.

[pdimitar]

> No training wheels.

Very cute for hobby projects, a huge liability for commercial projects.

Use as many training wheels there as humanly possible, please.

[sumanthvepa]

Sure. I use Java, Python and Javascript all the time. But when I need the performance, for demanding VR/ graphics, nothing comes close to combination of speed and expressive power of abstraction of C++.

[stocknoob]

Does a prion have a need to preserve itself?

If you make enough varied AIs, some will have self-replicating behavior, just like if you make enough random proteins, some will self-replicate.

[HKH2]

Does a prion think for itself? Who said self-replication is sufficient for AGI?

[pdimitar]

Oh yeah, it will replicate after the computer is shut down and then reinstalled from scratch. Especially when it's much simpler than that i.e. the whole thing lives in a throwaway container.

[rafaelmn]

> I really can't grasp how people think that a system that doesn't have a need to preserve itself will somehow start thinking for itself.

Society exists because cooperation outperforms the alternatives. If you have human level AI at some point there is no benefit to cooperation and a major incentive to prevent anyone else gaining access to equal/better AI.

AI itself does not need to have any motivation - people in charge have plenty of incentives to eliminate the rest once they don't need them anymore.

[HKH2]

Sure, in the prisoner's dilemma we could trust that all other parties will do the right thing, but that seems very unlikely.

[loki-ai]

that’s precisely the point: the technical geniuses lack creativity to predict how things can go wrong in a thousand of different ways.

[orbital-decay]

What makes you think they're predicting the apocalypse correctly, then?

Another thing the technical geniuses tend to be good at is exploiting the power they suddenly obtain in their own interest, either directly or with regulations and collusion with those who hold actual hard power.

Evil AI owners seem to be much closer and far more material than an evil AI, and coincidentally it's something that is almost entirely lacking from the discourse, as public attention is too focused on sci-fi hypotheticals.

[PeterisP]

The bar is different - saying "there is no risk of apocalypse" requires you to be ~100% certain, because it you're saying "I'm 99% certain that there won't be a an apocalypse" then you're on the side of the AI-risk people, because a low-probability extinction event does justify action; the risk argument isn't that apocalypse is certain but rather that it is sufficiently plausible to take preventive measures.

[nradov]

I am only 99% certain that we won't be invaded by hostile aliens. Therefore we should take measures like building a giant space laser to prevent that apocalypse.

[orbital-decay]

The issue is that the doomsday scenario is extremely vague. The actual mechanism of action of a hypothetical rogue AGI is usually handwaved away as "it will be self-improving, superhumanly persuasive, and far smarter than us, so it will somehow figure out how to do something, or convince us to do it". What exactly will happen? How exactly will it happen? Will the world do nothing until that moment? How do society, politics, military fit into that scenario? All that rationalist navel gazing I've seen so far is either hilariously unaware of the existence of the outside world or assumes it won't change in the process.

You can't fight what you can't even see, let alone not sure if it exists at all. You don't invent a pair of wings because 1900s' you thinks that "the scientists will invent an anti-aging cure in the next decade, and surely personal flight will be ubiquitous in 2000's". You don't design a plasma gun for your Mars landing just in case you land in a city between Martian canals and see an army of little green men there. The world doesn't work like that, by the time you reach the Mars surface the context will be wildly different. You get burned and put guardrails, maybe. Not the other way around. Nobody can see through higher order effects, no matter how smart they are. And as the threat becomes progressively more clear there will be more caution, if needed. Premature optimization yada yada.

What actually happens right now is everybody and my aunt seriously discussing the evil robots that will come and kill us. That's pure mass hysteria, caused by the scaremongering and the cult-like beliefs of very smart people with disproportional influence who can't contain their own conjectures and bullshit in the realm of science fiction.

On the other hand,the end goal of OpenAI is the major job replacement, according to their current charter. [1] "Broadly distributed"... will they distribute their utopia to North Korea? Not happening, isn't it? I think it's obvious that if the actual job replacement rate will ever get anywhere close to the levels of late 19th early 20th century industrialization, this will produce major societal shifts and struggles, wealth and power redistribution, and a lot of blood and wars. Because the dependence on your job is the only ephemeral influence you (as a worker) have on this world. And of course, the companies that control the AI will be gatekeepers, and they will be more than happy to close the open research and open source models, and pull the regulation ladder and lie in bed with politicians and military, like OpenAI already does for years, of course they realize that and their utopical self-contradicting "charter" is nothing more than marketing hogwash that they already changed and will change in the future.

This is far more realistic and will happen much earlier than the rogue AI science fiction, if happens at all. In fact it's slowly happening now, and it's not talked about nearly enough, because the attention is mostly misdirected onto the vague superhuman AI red herring.

[1] https://openai.com/charter

[oh_sigh]

So...who isn't lacking the creativity? In fact, I would say techies are unreasonably gloomy because they grew up obsessing over sci-fi.

[pdimitar]

Which also means they can't predict the apocalyptic scenarios.

Q.E.D.

[nsonha]

but all the thoughts about it in 1925 would have been way off to how it actually turned out

[calf]

I was thinking one reason Yann LeCun would make such a terrible analogy is because he knows something the rest of us don't.

[seanhunter]

The thing that he knows that (most of) the rest of us don’t is quite a lot about AI.

[throwaway09223]

If you read what Yann writes you'll pretty quickly see that he's rather ignorant about AI. His opinion is probably worse on average than the typical technical generalist's

[kelipso]

You'll have to be way more convincing than this if you want anyone to believe that about Yann haha.

[danielmarkbruce]

This is ignorant.

He won a Turing award for his work on deep learning.

Lots of people reasonably disagree with him about the future of AI/ML, but he's the opposite of ignorant.

[seanhunter]

That’s hilarious. I have read a few things he has written which suggests he’s definitely better than the average technical generalist. I haven’t read everything obviously but he has written quite a lot https://scholar.google.com/citations?user=WLN3QrAAAAAJ

[kranke155]

What an absurd idea to say this about a leading AI researcher

[thfuran]

And, what, you think he's working against humanity's interest in service of the secret AI overlords?

[calf]

You're missing the point and context. Person above me says this analysis is quite telling, then points out the counterfactual historical hypothetical, which makes no sense. Yann thinks supersonic flight is not worthy of precautionary principle ethics in 1925? I'm saying the same thing--Yann's terrible, nonsense analogy is indeed poorly argued, but plausibly would make sense as a Freudian slip inconsistency of some sort. Ergo, "it is quite telling". As to what contents in his mind, or his motivations, I don't care to speculate.

The fact that you make insinuations about what I think is pretty aggressive and terrible similarly, this forum ought to have better manners than that when writing replies to complete strangers. Not everyone who has a different opinion is some crypto conspiracy theorist, and you are wrong to jump to such a suggestion.

[ttoinou]

His funny insinuation pushed you to write a nice long argumented explanation, it worked great