[ben_w]

That's exactly wrong, we know some things that can be expressed as "a sequence of tokens" are harmful and indeed have already made them crimes.

What we need is to characterise what is possible so we can skip the AI equivalent of Union Carbide in Bohopal.

[ein0p]

Yes, but we also know that a knife can be used to slice vegetables or stab people, and we still allow knives. I can go to Google right now and easily find out how to make Sarin or ricin at home. Are you suggesting that we should ban Google Search because of that?

[ben_w]

> Yes, but we also know that a knife can be used to slice vegetables or stab people, and we still allow knives.

I'm from the UK originally, and guess what.

Also missing the point, given stabbing is a crime; what's the AI equivalent of a stabbing? Does anyone on the planet know?

> I can go to Google right now and easily find out how to make Sarin or ricin at home. Are you suggesting that we should ban Google Search because of that?

Google search has restrictions on what you can search for, and on what results it can return. The question is where to set those thresholds, those limits — and politicians do regularly argue about this for all kinds of reasons much weaker than actual toxins. The current fight in the US over Section 230 looks like it's about what can and can't be done and by whom and who is considered liable for unlawful content, despite the USA being (IMO) the global outlier in favour of free speech due to its maximalist attitude and constitution.

People joke about getting on watchlists due to their searches, and at least one YouTuber I follow has had agents show up to investigate their purchases.

Facebook got flack from the UN because they failed to have appropriate limits on their systems, leading to their platform being used to orchestrate the (still ongoing) genocide in Myanmar.

What's being asked for here is not the equivalent of "ban google search", it's "figure out the extent to which we need an equivalent of Section 230, an equivalent of law enforcement cooperation, an equivalent of spam filtering, an equivalent of the right to be forgotten, of etc." — we don't even have the questions yet, we have the analogies, that's all, and analogies aren't good enough regardless of if the system that you fear might do wrong is an AI or a regulatory body.

[ein0p]

What, you aren’t allowed to own kitchen knives? Or Google search somehow doesn’t return the chemical processes to make Sarin? Come on now.

[ben_w]

> What, you aren’t allowed to own kitchen knives?

You're not allowed to be in possession of a knife in public without a good reason.

https://www.gov.uk/buying-carrying-knives

You may think the UK government is nuts (I do, I left due to an unrelated law), but it is what it is.

> Or Google search somehow doesn’t return the chemical processes to make Sarin?

You're still missing the point of everything I've said if you think that's even a good rhetorical question.

I have no idea if that's me giving bad descriptions, or you being primed with the exact false world model I'm trying to convince you to change from.

Hill climbing sometimes involves going down one local peak before you can climb the global.

Again, and I don't know how to make this clearer, I am not calling for an undifferentiated ban on all AI just because they can be used for bad ends, I'm saying that we need to figure out how to even tell which uses are even the bad ones.

Your original text was:

> We know exactly what the system is capable of doing. It’s capable of outputting tokens which can then be converted into text

Well, we know exactly what a knife is capable of doing.

Does that knowledge mean we allow stabbing? Of course not!

What's the AI equivalent of a stabbing? Nobody knows.

[ein0p]

Same as you’re not allowed to commit eg election or postal fraud using LLMs. Are you allowed to carry a hammer? You can use that to kill people. You can also mow them down with a car, push them in front of a train with your bare hands, poison them with otherwise benign household chemicals and so on. It’s the applications that should be regulated, not the underlying tech