[zulban]

"These things should literally never be able to happen"

If we consider "humans using a bank website" and apply the same standard, then we'd never have online banking at all. People have brain farts. You should ask yourself if the failure rate is useful, not if it meets a made up perfection that we don't even have with manual human actions.

[aydyn]

Just because humans are imperfect and fall for scams and phishing doesn't mean we should knowingly build in additional attack mechanisms. That's insane. Its a false dilemma.

[wat10000]

Go hire some rando off the street, sit them down in front of your computer, and ask them to research some question for you while logged into your user account and authenticated to whatever web sites you happen to be authenticated to.

Does this sound like an absolutely idiotic idea that you’d never even consider? It sure does to me.

Yes, humans also aren’t very secure, which is why nobody with any sense would even consider doing this either a human.

[echelon]

The vast majority of humans would fall to bad security.

I think we should continue experimenting with LLMs and AI. Evolution is littered with the corpses of failed experiments. It would be a shame if we stopped innovating and froze things with the status quo because we were afraid of a few isolated accidents.

We should encourage people that don't understand the risks not to use browsers like this. For those that do understand, they should not use financial tools with these browsers.

Caveat emptor.

Don't stall progress because "eww, AI". Humans are just as gross.

We need to make mistakes to grow.

[saulpw]

We can continue to experiment while also going slowly. Evolution happens over many millions of years, giving organisms a chance to adapt and find a new niche to occupy. Full-steam-ahead is a terrible way to approach "progress".

[echelon]

> while also going slowly

That's what risk-averse players do. Sometimes it pays off, sometimes it's how you get out-innovated.

[Terr_]

If the only danger is the company itself bankrupt, then please, take all the risks you like.

But if they're managing customer-funds or selling fluffy asbestos teddybears, then that's a problem. It's a profoundly different moral landscape when the people choosing the risks (and grabbing any rewards) aren't the people bearing the danger.

[echelon]

You can have this outrage when your parents are using browser user agents.

All of this concern is over a hypothetical Reddit comment about a technology used by early adopter technologists.

Nobody has been harmed.

We need to keep building this stuff, not dog piling on hate and fear. It's too early to regulate and tie down. People need to be doing stupid stuff like ordering pizza. That's exactly where we are in the tech tree.

[forgetfreeman]

"We need to keep building this stuff" Yeah, we really don't. As in there is literally no possible upside for society at large to continuing down this path.

[wat10000]

This AI browser agent is outright dangerous as it is now. Nobody has been attacked this way... that we know of... yet.

It's one thing to build something dangerous because you just don't know about it yet. It's quite another to build something dangerous knowing that it's dangerous and just shrugging it off.

Imagine if Bitcoin was directly tied to your bank account and the protocol inherently allowed other people to perform transactions on your wallet. That's what this is, not "ordering pizza."

[girvo]

When your “mistakes” are “a user has their bank account drained irrecoverably”, no, we don’t.

[echelon]

So let's stop building browser agents?

This is a hypothetical Reddit comment that got Tweeted for attention. The to-date blast radius of this is zero.

What you're looking at now is the appropriate level of concern.

Let people build the hacky pizza ordering automations so we can find the utility sweet spots and then engineer more robust systems.

[wat10000]

1. Access to untrusted data.

2. Access to private data.

3. Ability to communicate externally.

An LLM must not be given all three of these, or it is inherently insecure. Any two is fine (mostly, private data and external communication is still a bit iffy), but if you give them all three then you're screwed. This is inherent to how LLMs work, you can't fix it as the technology stands today.

This isn't a secret. It's well known, and it's also something you can easily derive from first principles if you know the basics of how LLMs work.

You can build browser agents, but you can't give them all three of these things. Since a browser agent inherently accesses untrusted data and communicates externally, that means that it must not be given access to private data. Run it in a separate session with no cookies or other local data from your main session and you're fine. But running it in the user's session with all of their state is just plain irresponsible.

[ec109685]

The CEO of Perplexity hasn't addressed this at all, and instead spent all day tweeting about the transitions in their apps. They haven't shown any sign of taking this seriously and this exploit has been known for more than a month: https://x.com/AravSrinivas/status/1959689988989464889

[davidgerard]

> So let's stop building browser agents?

Yes, because the idea is stupid and also the reality turns out to be stupid. No part of this was not 100% predictable.

[recursive]

> So let's stop building browser agents?

The phrasing of this seems to imply that you think this is obviously ridiculous to the point that you can just say it ironically. But I actually think that's a good idea.