[goldenshale]

As if anyone is good at predicting the future. Please can we stop acting like expertise equates to fortune telling capabilities?! Nobody has any clue what a 1000x sized GPT model could do, and anybody who makes strong claims is a charlatan. In this age of paranoid AI risk cultists we need to cultivate humility and calm, a willingness to follow data rather than beliefs and predictions.

[p-e-w]

> paranoid AI risk cultists

There is broad consensus among experts that a hypothetical strong AI would be a threat, and potentially an existential threat, to humanity. While not everyone agrees on details like timeline and alignment issues, the idea that AI is dangerous is not a cult, it's the mainstream view.

Climate scientists cannot "predict the future" with certainty either. That doesn't mean their warnings are hot air, and neither are the warnings from AI safety experts. It seems like the educated masses are currently in denial about AI in much the same way as the uneducated masses have been in denial about climate change for a while.

Risk assessment doesn't require understanding. I don't have to understand how a venomous snake senses prey in order to know that the snake is a potential threat to me. In fact, the less I know about the snake, the higher the assessed risk should be, since the uncertainty is higher as well.

[goldenshale]

What nonsense. I've spent over a decade 100% focused on AI, and the broad consensus among everyone I've worked with is not to be that concerned at all. The only consensus is that a small group of self proclaimed experts who make a lot of noise is that they get lots of press coverage if they scream and shout making predictions based on zero scientific evidence.

We can understand the physics of greenhouse gases and take measurements of earth systems to build evidence for models and theories. (Many of which are nonetheless very inaccurate beyond short time horizons.) Show me any evidence for AI risk today beyond people's theories and beliefs?

The best predictor of the future is the past, not people's wild ideas about what the future could be. I'm not about to sit here feeling scared because there is more uncertainty that our matrix multiplies are about to go rogue. There are no AGI experts or AI risk experts, because we don't have any of these systems to study and analyze. What we have is people forming beliefs about their own predictions about systems which are unknowable.

[p-e-w]

> Show me any evidence for AI risk today beyond people's theories and beliefs?

Deduction. Empirical evidence isn't the only source of insight. You don't have to conduct experiments in order to reasonably conclude that an entity that

1. outperforms humans at mental tasks

2. shares no evolutionary commonality with humans

3. does not necessarily have any goals that align with those of humans

is a potential threat to humans. This follows from very basic deductive analysis.

> There are no AGI experts or AI risk experts, because we don't have any of these systems to study and analyze.

Indeed. Which increases the risk. Unless you are claiming that AGI is actually impossible, the fact that its properties and behavior cannot be studied should make people even more worried.

Uncertainty and lack of knowledge are what risk is. How little we know about potential AGI is exactly why AGI represents such a big risk. If we completely understood it and were able to make reliable predictions, there would be zero risk by definition.

[goldenshale]

1) Computers, smart phones, and pocket calculators also outperform humans at mental tasks. So do birds, dolphins, and dogs for that matter, at tasks for which they are specialized.

2) so? What are you imagining this implies? An infinity of possibilities does not a reason make, unless you are talking about arbitrary religious beliefs.

3) Right, no goals, no will, no purpose. Just some matrix multiplies doing interesting things.

Deduction requires a premise which then leads to another premise or a conclusion due to accepted facts or reasons. I'm genuinely curious why you think any of these properties automatically implies danger?

The future is uncertain. The stock market, the economy, your health, your friendships and romances, are all unpredictable and uncertain. Uncertainty is not a reason to freak out, although it might encourage us to find ways to become adaptable, anti-fragile, and wise. I think AI will help us improve in these dimensions because it is already proving that it can with real evidence, not beliefs.

[tim333]

It seems a safe prediction if you extrapolate that AI will get generally smarter than humans.

It also seems a safe prediction, given past human behaviour that some humans will set some AI to do bad stuff.

Therefore risk.

(eg "chat gtp 27, help me make billions on crypto and use it to set up a distributed army to take over the world")

[doctor_eval]

> chat gtp 27, help me make billions on crypto and use it to set up a distributed army to take over the world

Yevgeny Prigozhin has entered the chat.

[b1daly]

the world is incredibly filled with risk to humans—people in the AI doomer camp are making a claim that AI potentially is a new kind of uncontrollable risk that warrants extraordinary regulation

the basis of this claim seems to be a confusion of logical or deductive reasoning with inductive or observational reasoning

argument comes down to

- it’s possible to imagine a super intelligent machine that has properties that will kill everyone (this is an exercise in logical reasoning)

- since it’s possible to imagine it, this means it will come into existence — this is an error because things that exist in the real, physical world do so based on physical processes governed by inductive reasoning

generally, there is a long series of steps between the imagining of some constructed, complex machine and its realization, along with its conceptual foundations it requires sustained effort, trial and error, maintenance, generally a serious fight against entropy to make it function and keep it functioning

the sort of out of control AI imagined by AI doomers is not something we’ve seen before

so we shouldn’t make costly decisions based upon this confusion of reasoning

[p-e-w]

> since it’s possible to imagine it, this means it will come into existence

Nope. That's not the argument. In fact, it's such a bad take that it reeks of a deliberately constructed strawman.

The actual argument is: Since it's possible to imagine it, and doesn't contradict any known laws of nature or technology, and current development appears to be iterating towards it, it might come into existence, thus it presents a statistical risk.

When I take out tornado insurance, it's not because I know my house will be blown away by a storm – it's because I don't know, but the possibility is there.

Certainty is not required in order to conclude that risk exists. Quite the opposite is true: Risk is a function of uncertainty.

[anon373839]

The word “entity” is doing some quiet but heavy lifting here. I think it would be a good idea to specify what you really mean by this term, and how we can logically deduce the development of such a thing from existing technology (deep learning).

[Tenoke]

>I've spent over a decade 100% focused on AI, and the broad consensus among everyone I've worked with is not to be that concerned at all.

I also work in AI and I don't mention my concerns to colleagues who are so anti-AI risk as you. Perhaps your ideas about your colleagues' views are distorted.

[literalis]

IMO it is really just paranoid AI risk cultists theater for narcissists.

The more narcissistic types have figured out it is their moment in the sun to see their name in the paper and the more they play up the idea that AI is going to eat us the more attention they will get from media.

The whole idea is so irrational that I fail to see what other explanation there really is.

The other guilty party are the masses that have been trained to think in terms of appeal to authority instead of using their own brains. They have created the audience for this theater.

[ChatGTP]

I don't think it's wise to just give this one the climate change treatment, that is not listening to the scientists and not taking action or taking it seriously until it's a catastrophe.

[civilized]

"Expertise" in a speculative concept like AI risk is not remotely comparable to expertise in a scientific field like climate change.

There are two definitions of expertise:

1. Knowing more than most people about a topic. This is the type of expertise that wins the Quiz Bowl.

2. Actual mastery of a field, such that predictions and analyses generated by a person possessing such mastery are reliable. This is the type of expertise that fixes your home or car.

The first definition is easily verifiable, and due to the availability heuristic, it is often presented as a legitimate proxy for the second. But it isn't really, not in general.

If I know more about horoscopes than most people, I am a horoscope expert. But it doesn't mean I can be relied on to predict any of the things horoscopes supposedly predict. It's the same with AI risk. Expertise in AI risk is not a basis for credibility because AI risk is not a real scientific field.

Climate change is a real field of science. AI risk is Nostradamic prognostication by people who know more than you.

[p-e-w]

> Climate change is a real field of science. AI risk is Nostradamic prognostication by people who know more than you.

Any prediction of the future is necessarily based on modeling and extrapolation.

Five years ago AIs couldn't pass a third-grade reading comprehension test. Today they pass in the top 10% of law, medical, and engineering exams for human professionals.

It is absolutely possible to extrapolate from such developments, and doing so is scientific, not "Nostradamic". Many predictions of the potential impact of climate change also include speculative elements, such as societal effects, migration patterns, conflicts, etc., which cannot be modeled or forecast with any real certainty. That doesn't make them unscientific.

[civilized]

It's possible to extrapolate from a horoscope too, it's just not that useful. Let's talk about the massive difference in the extrapolation being done here, climate change vs AI.

In climate change, we are analyzing historical climate data using weather models representing known physical processes. We try to predict the data using these models, and we are only able to do so if we include the forcing from greenhouse gases. From this we can constrain the range of impacts these gases could be having on temperature and forecast likely futures. The forecasts are heavily informed by a thoroughly validated base of prior knowledge, not just drawing lines through a log log plot.

None of this has any counterparts in AI. We don't understand AI systems to anywhere near the level that physics affords understanding of physical systems. We don't even understand them at a Moore's Law level, where you can at least know what engineering innovations are in the pipeline and how far they could plausibly go. Predicting the sophistication of future AI is just Nostradamic prognostication.

Yann LeCun recently gave a presentation arguing that LLMs are a dead end and proposing a completely different approach. His arguments were extremely heuristic and unconvincing, but this at least shows that both sides have bigwigs with unconvincing heuristic arguments.

[antifa]

> a hypothetical strong AI would be a threat, and potentially an existential threat, to humanity.

I wish a cool scifi robot woke up one day and violently optimized all of humanity into paperclips, instead I live in the real world where the jobs are going to evaporate like water in a newly installed desert and the "let them eat cake" will get increasingly louder and blue-check-markier.

[ilaksh]

Warning people about potential extreme risks from advanced AI does not make you a cultist. It makes you a realist.

I love GPT and my whole life and plans are based on AI tools like it. But that doesn't mean that if you make it say 50% smarter and 50 times faster that it can't cause problems for people. Because all it takes is systems with superior reasoning capability to be given an overly broad goal.

In less than five years, these models may be thinking dozens of times faster than any human. Human input or activities will appear to be mostly frozen to them. The only way to keep up will be deploying your own models.

So to effectively lose control you don't need the models to "wake up" and become living simulations of people or anything. You just need them to get somewhat smarter and much faster.

We have to expect them to get much, much faster. The models, software, and hardware for this specific application all have room for improvement. And there will be new paradigms/approaches that are even more efficient for this application.

For hyperspeed AI to not come about would be a total break from computing history.

[goldenshale]

A realist is someone who accepts reality as it is, not as they might be able to anxiously envision that it could be. Life is too short and attention too precious to fill the meme space with every dreamer's deepest concerns. None of these dramatic X-risk claims is based on anything but beliefs and conjecture. "Thinking dozens of times faster?" What do you even mean? These are models executing matrix multiplies billions of times faster than our brains propagate information, and they represent knowledge in a manner which is unique and different from human brains. They have no goals, no will, and no inner experience of us being frozen or fast or anything else. We are so prone to anthropomorphize willy-nilly. We evolved in a paradigm of resource competition so we have drives and impulses to protect, defend, devour, etc., of which AI models have zero. Anyone who has investigated reinforcement learning knows that we are currently far away from understanding let alone implementing systems which can effectively deconstruct abstract goals into concrete sub-tasks, yet people are soooo sure that these models are somehow going to all of a sudden be an enormous risk. Why don't we wait until there is even the slightest glimmer of evidence before listening to these prophets of doom?

This pseudo-intellectual belief structure is very cult like. Its an end of the world scenario that only an elite few can really understand, and they, our saviors, our band of reluctant nerd heroes, are screaming from the pulpit to warn us of utter destruction. The actual end of days. These "black box" (er, I mean, we engineered them that way after decades of research, but no, nobody really understands them, right?) shoggoths will be so incredibly brilliant that they will be able to dominate all of humanity. They will understand humans so well as to manipulate us out of existence, yet they will be so utterly stupid as to pursue paper clips at all cost.

Maybe instead these models will just be really useful software tools to compress knowledge and make it available to humanity in myriad forms to develop a next level of civilization on top of? People will become more educated and wise, the cost of goods and services will drop dramatically, thereby enriching all of humanity, and life will go on. There are straighter paths from where we are today to this set of predictions than there are to many of the doomsday scenarios, yet it has become hip among the intelligentsia to be concerned about everything. Being optimistic is somehow not real, (although the progress of civilization serves as great evidence that optimism is indeed rational) while being a loud mouthed scare mongerer or a quiet, very serious and concerned intellectual, is seen as respectable. Forget that. All the doomers can go rot in their depressive caves while the rest of us build a bad ass future for all of humanity. Once hail bop has passed over I hope everyone feels welcome to come back to the party.

[siilikuin]

Let's try to rewrite this in a somewhat more dispassionate style:

A pragmatic perspective requires one to accept the present reality as it is, rather than hypothesize an exaggerated potential of what could be. Not all concerns surrounding existential risks in technology are necessarily grounded in empirical evidence. When it comes to artificial intelligence, for instance, current models operate at a speed vastly superior to human cognition. However, this does not equate to sentient consciousness or personal motivation. The projection of human traits onto these models may be misplaced, as AI systems do not possess inherently human drives or desires.

Many misconceptions about reinforcement learning and its capabilities abound. The development of systems that can translate abstract objectives into detailed subtasks remains a distant prospect. There seems to be a pervasive certainty about the risks associated with these models, yet concrete evidence of such dangers is still wanting.

This belief system, one might argue, shares certain characteristics with a doomsday cult. There is a narrative that portrays a small group of technologists as our only defense against a looming, catastrophic end. These artificial intelligence models, which were engineered after extensive research, are often misinterpreted as inscrutable entities capable of outsmarting and eradicating humanity, while simultaneously being so simplistic as to obsess over trivial tasks.

Alternatively, these AI models could be viewed as valuable tools for knowledge compression and distribution, enabling the advancement of civilization. As a result, societal education levels could improve, and the cost of goods and services might decrease, which could potentially enrich human life on a global scale. While there seems to be a tendency to worry about every potential hazard, optimism about the future is not unfounded given the trajectory of human progress.

There are certainly different perspectives on this issue. Some adhere to a more fatalistic viewpoint, while others are working towards a brighter future for humanity. Regardless, once the present fears subside, everyone is invited to participate in shaping our collective future.

[goldenshale]

Hahaha, thanks ChatGPT! This is better said than my snarky, frustrated at the FUD version, and I can learn from the approach.

[Applejinx]

No, it's really not, because your riff on 'shoggoths that are both so brilliant as to be dangerous, yet so stupid that they maximize paperclips' touches on an important point that the summarized version completely omits.

AI is exactly that kind of stupid. What it lacks isn't 'brilliance' but intentionality. It can do all sorts of rhetorical party tricks, including those that are good at influencing humans, it can even very likely work out WHICH lines of argument are good at influencing humans from context, and yet it has no intentionality. It's wholly incapable of thinking 'wait, I'm making people turn the world to paperclips. This is stupid'.

So it IS likely to turn its skills to paperclip maximization, or any other hopelessly quixotic and destructive pursuit. It just needs a stupid person to ask it to do that… and we're not short of stupid people.

So what you said was better, snark and all :)

[ilaksh]

Not sure you read my comment carefully enough. I am an optimist. I do believe that AI can and probably will be a positive and transformative force.

But I also think it's more anticipatory than speculative to envision AI systems (quite possibly on the request of a human faction) taking control.

And GPT-4 absolutely does do abstract reasoning and subgoals. No it doesn't have many other capabilities or characteristics of humans or other animals but as I said it doesn't need those to be dangerous.

We need to prohibit manufacture or design AI hardware that has performance beyond a certain level. It is not too early to start talking about a risk that could end humanity. I do hope that we can get away with something a few orders of magnitude better than what we have today, but it's really of asking for trouble the more we optimize it, and we may be walking a fine line within a decade or so. Or less. It takes years to design hardware and get manufacturing online, especially for new approaches.

And two orders of magnitude faster may be only a few years away.

[ignoramous]

> As if anyone is good at predicting the future.

Whoever predicts the right direction, (and when the time is right) puts money where their mouth is, stands a shot at unseating... the alt man.

  I think the way to use these big ideas is not to try to identify a precise point in the future and then ask yourself how to get from here to there, like the popular image of a visionary. You'll be better off if you operate like Columbus and just head in a general westerly direction. Don't try to construct the future like a building, because your current blueprint is almost certainly mistaken. Start with something you know works, and when you expand, expand westward.
  
  The popular image of the visionary is someone with a clear view of the future, but empirically it may be better to have a blurry one.

paulgraham.com/ambitious.html

[JimtheCoder]

Sir, this is a discussion forum...