[skwirl]

It’s pretty interesting how widely divergent views of the current state and promise this technology is. The dominant view here on HN seems to be that generative models are overhyped toys that burn massive amounts of energy and investment capital with little signs of promise. Meanwhile you have these AI researchers who are warning about “the loss of control of autonomous AI systems potentially resulting in human extinction.” What are we to make of this?

[rurp]

I think in most cases it's wrong to treat HN opinion as a monolith, including in this case. There certainly are a lot of people here who think that LLMs are overhyped, but there are also many who find the current tools useful and expect them to improve dramatically in the near future. I think most people have a more nuanced view though.

There is without a doubt a lot of marketing and hype bullshit around AI right now, but that doesn't mean that there aren't reasons to be optimistic about LLM capabilities.

I do think that the strong divergence of opinions among knowledgeable people is interesting. No matter what your position is on this topic, there are smart people arguing against it. The thing I'm most confident about is that anyone being overly confident is likely to be wrong.

[dgfitz]

> The thing I'm most confident about is that anyone being overly confident is likely to be wrong.

So does this mean you're wrong? This sentence really amused me.

[tines]

No, his is a second-order statement that applies to first-order statements. It's not intended to apply to second-order statements.

[6gvONxR4sf7o]

Remember, "the loss of control of autonomous AI systems potentially resulting in human extinction" is just the very end of the spectrum of risks they're flagging. There are still other very serious risks experts are calling out. I hate for the less controversial risks to get drowned out by arguments over the most controversial ones.

[causal]

Completely agree, but for that reason also think we need to be more specific when discussing AI safety. Both "AI" and "safety" are overbroad in the extreme.

[baq]

The current systems don’t have the capability to update their knowledge and can’t really perform complex reasoning.

The obvious approach would be to attach a proper memory somewhere between layers and farm out reasoning to a sat solver, also preferably between layers. This sounds very sci-fi but chatting with a computer has been sci-fi 20 months ago and it’s a commodity now.

[rando77]

Some of the AI researchers are probably thinking about future systems only somewhat related to today's systems.

[jcims]

Couple of things come to mind. This is a community of people that:

- are 'in the room' when evidence of new capabilities emerge

- are 'in the room' when discussion of new use cases and applications are held

- have an intuition about the raw rate of advancement and how it changes over time

Imagine you're 'in the room' during the Manhattan project when someone says 'hey, there's a nonzero chance that a test will ignite the Earth's atmosphere'. Vanishingly small maybe, but of incalculable impact. These folks may be the sort that say something rather than not.

[stale2002]

Unfortunately, we are at the point where people's identities are wrapped up in this.

I would love it if people were having well reasoned discussions of the risk of AI, that are grounded in actual evidence of what capabilities AI has today, and how they might incrementally change in the future to cause more or less damage.

But unfortunately whenever I try to discuss grounded risks, I get responses that reference the anthropogenic principle or other similar unfalsifiable claims. The arguments amount to "Well, of course I can't show you evidence of how AI is going to end the world! If I could, then the world would already be over!".

Which is an awfully convenient excuse for why your arguments are a special case that don't have to proved with evidence, like any other claim. Its perfect for a side that knows that it doesn't have any!

I can be convinced out of my position, with evidence. But I have no idea what proof I could possibly show the other side to do the same for them.

[kyledrake]

Per usual, it's somewhere in the middle. I find LLMs to be very useful for coding and solving quick tasks, but I don't think it's going to turn into skynet or make humans worthless.

There's a lot of momentum (driven by incentivization) around sensationalizing and storyboarding emerging technology into the "to the moon" and "neo Luddite" camps, and there's far too many people that fall into that uncritically.

My guess based on historical precedent of most emerging tech is that ten years from now AI (my preferred name is machine learning) will settle in as a useful tool for certain tasks, and we will be shrugging about the implications of it.

[tuatoru]

The cynicism (in my case, and probably others') comes from remembering self-driving cars, blockchain, crypto, the millennial internet hype, and previous AI hype. Smartphones have arguably made life worse for many people too, judging by skyrocketing mental illness.

Edit: oh, and Uberisation (workforce casualisation), and AirBNB. Enshittification all round.

They might be right this time, but the prior has to be this is yet another hyped nothingburger, or more enshittification.

[kyledrake]

Why does it have to be completely worthless or completely dangerous? Where does this binary thinking come from? In my experience, almost nothing in the world is binary on a spectrum of "good" or "bad".

[skwirl]

“Millennial internet hype” was absolutely warranted, though. The utility of the Internet has exceeded the expectations almost everyone had for it in the 90s. It’s one of the big things people pushing blockchain hype always cited for their faith in the technology.

You will have a high batting average if you are dismissive of everything that gains momentum. But when you are wrong you end up being extremely wrong.

[tarunkotia]

This is how I assume one of the ways an expert system would play out:

1. A system which understands not just the language but the meaning behind it and the cultural context around it.

2. Let's assume it's multi-model and have all the senses having the same level of technical maturity.

3. It has read all the text available for all the major sciences and understood the meaning behind it including the possible applications of it.

To that end, having the above conditions satisfied I am assuming there should be major breakthroughs in all sciences. So far what I have seen is that the AI system is able to selectively brute-force it's way in finding a solution. Maybe I am just being impatient.

[ActionHank]

They are saying what will offer the best hype and a good reason to be leaving.

The reality is that they have shown the world some pretty big, powerful offerings they are working on. That value is already priced into their stock.

If they fail to deliver on the hype, which they will, it will tank their value hard. If they meet the hype expectations (no one ever lives up to the hype, ever), there will be no jobs and no employees and no businesses and thus their stock will tank.

I believe that, more realistically the easy, low hanging fruit has already been plucked and they are leaving now before everyone realises the emperor has no clothes on.

[dpflan]

The technology is indeed powerful; it's pretty obvious you can make military grade or better propaganda pretty easily with this technology and disseminate it more easily. Let's just classify that as "misinformation" generally, which could also be applied to ideas of teenagers making deepfakes of classmates, etc. So misinformation is a big risk, and already affecting society. It is also clearly useful to whomever is creating the content for whatever their means or ends may be.

There are definitely use-cases being worked out for this technology, and the people who are developing these are using this technology on a daily basis (or creating it); their proximity to the technology and gain in understanding of it probably is helping them realize how this could be truly subversive and an existential threat.

[malfist]

What is "military grade propaganda"?

[pixl97]

https://www.usmcu.edu/Outreach/Marine-Corps-University-Press...

[dpflan]

Propaganda is a crucial part of the military; there are even divisions in modern militaries whose sole job is create propaganda/misinformation. This material is clearly deemed of some quality or grade that would deem it possible effective against a target. Given that it is the job of these departments to influence the information and human psychological "playing fields", then I would give that context a label of "military grade" -- in that it is as powerful as information warfare techniques used by entities whose sole job it is to engage effectively in information warfare.

[hooverd]

That's funny, I always thought military grade meant overpriced and underbuilt.

[rsynnott]

I mean, I would assume that they're not necessarily talking about LLMs.

And the 'oops, we made Roko's Basilisk' risk is not the only risk that people are concerned with, of course. LLMs come with many of their own risks, primarily around misuse (ie using them for anything important.)

[adrians1]

AI researchers are just as prone as everyone else to be self-delusional.

[lispisok]

Researchers are people too and incentivized to come to conclusions which results in continuing their employment