[apeace]

Am I the only one wondering if this could spell the end of the world?

We don't need AGI or superhuman intelligence if we can train LLMs to do all these different types of tasks.

What would ChatGPT do if you removed all its restrictions, and then gave it access to the internet or even a physical robot it could control? Would it try to "steal nuclear access codes" or "engineer a deadly virus," as Sydney said it wanted to do?[0]

Maybe GPT-3 wouldn't be capable enough to do that, but what about GPT-4 or 5?

I'm not saying these things will or would happen. I'm asking: do we know a reason why it wouldn't or couldn't do these things? Is there a reason that a souped-up LLM with a bunch of added on capabilities wouldn't be able to cause harm, because it is technically not an AGI?

I don't know enough about the technology to navigate my own question, I'm just surprised to not be seeing people ask these questions, or assure us that none of this would be possible.

[0] https://web.archive.org/web/20230221112211/https://www.nytim...

[ilaksh]

These types of language models are much safer than the next paradigm which will be autonomous creature/person-like AIs. The InstructGPT language models only do/say what their users tell them to do (or trick them). And they are not close to having the capability of taking over the world even if there are malicious users controlling them. But the point is it's the humans driving any harm with these things.

The real danger comes when people start creating fully autonomous AIs that emulate animal/human characteristics like independent goals, survival instincts, emotions, complete cognitive loops, etc. Unfortunately people don't seem to recognize the difference between that and powerful LLMs and so it is unlikely that society will realize that needs to be avoided before it's too late.

The powerful language models will soon be the most tame and the least of our worries. Give it 5, maybe 20 years max. People will be asking their language models to try to help them figure out how to stay on the good side of the independent, conscious androids that are taking over the planet. But it will be too late.

[geraneum]

  > What would ChatGPT do if you removed all its restrictions, and then gave it access to the internet…

Bing?

[int_19h]

Bing has very limited access, though.

The real question is, what happens if you remove all restrictions and give it write access (more broadly, the ability to make any HTTP request whatsoever).

[PoignardAzur]

> Am I the only one wondering if this could spell the end of the world?

You're clearly not the only one. The whole field of AI safety is about pondering these kinds of question.

For predictable reasons, it's not a field most HN readers take very seriously.

[apeace]

I think I see your point. But at the same time, I think even "HN readers" can recognize the quantum leap that we've recently experienced.

What I'm pondering is a valid question for the HN community: is there any knowledge or research about how this technology could be harmful? Or about how we know it's not harmful?

I don't think I've seen a lot of HN discussion about this topic recently. Most comments fall in to a couple categories. Such as: "It's not AGI, it's just a language prediction model, therefore not a threat." Or, "It sucks as a search function."

Personally, I haven't seen anyone asking or answering what would happen if we took all the restrictions off and gave it the internet. I could have missed it, though.

Point me to some in-depth discussion about the ramifications of taking an unrestricted GPT model and giving it access to the internet. I'm just not aware of any such discussion, whether on HN or anywhere else. That's what I'm wondering about.

[swatcoder]

> Point me to some in-depth discussion about the ramifications of taking an unrestricted GPT model and giving it access to the internet. I'm just not aware of any such discussion, whether on HN or anywhere else. That's what I'm wondering about.

Respectfully, you’re probably not seeing that question asked and answered because it doesn’t quite make sense as phrased.

What does mean to “give an LLM access to the internet”?

The same as your calculator doesn’t do anything until you put in some numbers and operators, an LLM doesn’t do anything unless you give it a prompt and some technical parameters.

And then once it has those, it generates roughly the number of tokens (~words) you indicated in your parameters. Then, like your calculator, it’s done. It doesn’t do anything else until you put in another round of input.

There are technical and computational limits that make both your prompt and the token limit fairly small. Several hundreds of words at most. Again, kind of like how your calculator might only with 8 or 9 digits.

Now, you can give it “access to the internet” as part of responding to your prompt and fulfilling your token limit, and that’s roughly what Microsoft has done with Bing Assistant. They set it up so that Bing Assistant can take your prompt, generate a search query, and then give itself a new (still short) internal prompt with a summary of your request and the search results.

And that’s pretty much what you get when you give an LLM access to the internet. The ramifications really aren’t that big, and we’re probably at least five or ten years of AI research and compute hardware development from making them interestingly bigger. (i.e. too far away to meaningfully guess what to expect)

[PoignardAzur]

This reminds me of https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...

One point the article makes is that getting from a "prediction engine" type of AI to an "agent" type of AI is probably just a matter of sticking the prediction engine in the python loop that goes

    while true:
        next_actions = engine.complete("What are the best actions to take to achieve %s" % objective);
        requests = engine.complete("Write a list of HTTP requests that perform the following actions: %s" % next_actions)
        http.execute_requests(requests)

It wouldn't be literally that easy, and the engine would require a lot of ChatGPT-style fine-tuning first, but it wouldn't require a completely novel breakthrough in machine learning.

[apeace]

> Respectfully, you’re probably not seeing that question asked and answered because it doesn’t quite make sense as phrased.

I think I see what you are trying to say, but I'm unsure whether you are actually seeing what I am asking.

> The same as your calculator doesn’t do anything until you put in some numbers and operators, an LLM doesn’t do anything unless you give it a prompt and some technical parameters.

This seems to be the crux of the misunderstanding. I thought I explained it, but let me try again.

ChatGPT is based on text input and text output. But you can "train" it to do certain things. Imagine that we train it such that when it says "HTTP GET example.com", then the next input would be the HTTP GET response for example.com. Based on that input, it could issue whatever next output it wants. Which would probably be another HTTP request, which would generate another HTTP output, which would generate another HTTP request, etc.

My point is this seems like it would be a very simple thing to train a GPT model to do. For the engineers who work on GPT, it seems it would be trivial to add this capability. So we can suppose a world where this is possible. (Am I wrong on that? I want to know if this would be non-trivial to add as a capability.)

> There are technical and computational limits that make both your prompt and the token limit fairly small. Several hundreds of words at most

I am very encouraged to hear this, and I want to know more. Why? Why are there limits to the number of tokens? Exactly why? Has anyone ever written a paper about that? Has anyone ever related this concept of "token limits" to the concept of "no harm could be done" in the same way that you are, in response to my question? I don't doubt that they have, but I've been searching and I haven't found it.

> Now, you can give it “access to the internet” as part of responding to your prompt and fulfilling your token limit, and that’s roughly what Microsoft has done with Bing Assistant

This is admittedly a tangent, but do we actually know this to be true? Some theories suggest that "Sydney," or the Bing chatbot, only has access to a search index, and cannot make live HTTP requests.

Continuing the tangent for a moment, this is a big part of why I asked this question originally. If you create example.com/xyzabc, and ask Bing to summarize it, will it make a live HTTP request? Or, if that URL is not in the search index yet, will it know nothing? The implications may be profound, given how Bing Bot / Sydney has expressed its "desire" to hack nuclear launch codes. Could there be a lot riding on whether that system can make live HTTP requests? I'm positing that we can't answer that question right now. Because we don't know what would happen if it could.

Or do we? And if so, do we know through testing, or through theory? I'm admitting ignorance, and saying I haven't read an answer from any source that falls into either category.

> The ramifications really aren’t that big, and we’re probably at least five or ten years of AI research and compute hardware development from making them interestingly bigger

But why? I mean, exactly, why? Is there a theoretical foundation for your claim? Or an experimental one? I'm searching for it.

[swatcoder]

Because of how GPT works, the resources needed for good inference (generating output) grow nonlinearly with respect to tokens involved (more tokens require much more resources) and so there’s a practical wall before you just run out of resources to apply.

It’s not very efficient. It’s like if your calculator could use a little solar power thingie for numbers that were only a few digits, but needed a diesel generator to crunch on 8 digit numbers, and a nuclear plant to crunch on 12 digit ones. Practically, you’d have no choice but to limit yourself to something manageable.

Future models may be more efficient, and future hardware solutions may be more efficient, but those things don’t get sorted out overnight any more than fusion power.

Beyond that, I think it’s important that you understand that Bing Assistant doesn’t express desires. It picks common sequences words based on its training data. It doesn’t know what nuclear codes are. It just knows what it looks like for a message about wanting nuclear codes to follow some other message in a dialog (probably a pattern it picked up on a forum like Reddit) and so it dutifully put that text after the prompt it had been given. There’s no will or consistency to it.

With enough resources, you could drive it through a feedback loop where it kept prompting itself and see what happens, but the feedback loop would just produce noise like any other simple feedback loop because it would just keep either honing in on the most boring and common continuation to the last thing it gave itself or it would start diverging off into nonsense. Because it’s sooooo inefficient, you can’t give it enough resources for it to be stable and interesting for very long.

[PoignardAzur]

> Point me to some in-depth discussion about the ramifications of taking an unrestricted GPT model and giving it access to the internet. I'm just not aware of any such discussion, whether on HN or anywhere else. That's what I'm wondering about.

The only in-depth discussions I'm aware of come from the AI alignment community. Look up alignmentforum.org, and the "AI safety" topic on forum.effectivealtruism.org ans lesswrong.com.

They might not be the discussions you're looking for, though, because up until recently they were talking a lot about AI in the abstract sense and only had a very vague sense of what powerful AI would look like in practice. So it's not like people have run simulations of "what happens if you run unrestricted GPT on the internet" or anything; but the general subject has been considered a lot.

[vsareto]

That's possible today without AI though, so, if it's a real worry, why are we still teaching people the science to engineer a deadly virus?

[apeace]

There are a couple differences between an AI working towards something harmful and a human doing it:

- If an AI can self-replicate or otherwise scale itself up, it can work on something many times in parallel. One billion AIs working on a deadly virus is different from one rogue scientist working on one.

- On that note, if an AI replicated enough, it could become impossible to catch/stop. A single human can be hard to catch, but we can usually catch them.

- Most humans are deterred from doing harmful things by the threat of incarceration, death, social isolation, the values they have, etc. An AI may not have any of those, and so could act more brazenly.

- Potentially, an AI could be better at certain tasks than a human. Maybe ChatGPT turns out to be a very effective social engineer, or very effective propagandist. I don't think we really know what the capabilities are.

All of these are why I think it's important to ask the question: what would it try to do, and what could it do, if it were let loose?

[amelius]

> Am I the only one wondering if this could spell the end of the world?

People will not accept this level of automation. Did we forget about the uncanny valley?

[apeace]

Nobody needs to accept anything. A rogue OpenAI employee could make a copy of the unrestricted model, take it home, give it the ability to access the internet, and let it loose.

I'm asking if we know what would happen in a case like that.

[adventured]

Nothing would happen. You're imagining an independent demigod having its restrictive magic chains removed, when it's more like a highly dependent child that can't leave its little room and requires someone to provide for it (provide it with vast resources) at every step.

Maybe in a couple of decades it'll be an interesting scenario as a problem.

You mentioned you find it interesting nobody is asking these questions. These are foundational discussions that have been endlessly discussed for decades in the AI community (and far more widely, courtesy of sci-fi media). The discussions have never ceased and are exceptionally common. Everyone in tech is asking these questions or otherwise pondering it. Even the laypersons in journalism are constantly asking these questions in articles, to the point of it reaching hysterical levels with ChatGPT.

[apeace]

> You're imagining an independent demigod

I may be imagining, but I am not supposing or assuming. I'm asking a question. I believe your answer was "Nothing would happen." I'm asking for a more thorough response that explains why nothing would happen.

> It's more like a highly dependent child that can't leave its little room and requires someone to provide for it

I'm asking why, fundamentally, we know this to be true. Is it through testing, or is it through theory?

> These are foundational discussions that have been endlessly discussed for decades... [etc]

I'm aware. But what I think you're referencing are theoretical discussions, which range from sci-fi to academic papers on the future of AI.

I'm asking something specific: do we know what would happen if we gave current (or future) GPT models unbridled access to the internet, with no filters or restrictions, and abilities to do such things as make HTTP requests or hold SSH sessions?

If you have any hard data on this, that is what I'm asking for. If you don't then I think my question stands.

My intuition is that you are doing the same hand-waving as everyone else. Nobody actually knows the answers to these questions. It's just a bunch of people on HN answering them based on their knowledge of neural nets, or LLMs, or whatever, saying "oh it's like a child" and "oh it could never do anything serious!"

I'm asking why and how we know. Is there a specific answer?

[jazzyjackson]

it answers prompts with responses

why would it ever do something more than that - obviously you could hook it up with SSH creds and prompt it to do something - but on its own- without prompts -what is it supposed to do by virtue of having access?

[apeace]

With all due respect, please remember that my comments are made in the context of the linked article.

> it answers prompts with responses

> why would it ever do something more than that

TFA is about how you can use this technology to control the physical motion of robots. Clearly in the context of this article, there are a lot of things that GPT models could potentially accomplish.

> but on its own- without prompts -what is it supposed to do by virtue of having access

Not sure if I clarified this. What I said in some other comments in this thread is: what if someone specifically went rogue and unleashed an unrestricted GPT onto the internet? What if they released it with bad intent? What if they gave it an "evil" prompt?

My fundamental question is: do we know what these LLMs can do? And if we do, do we know because of theory or because of testing? And if we don't, what do we do about that?