Hacker News new | ask | show | jobs
by saurik 457 days ago
FWIW, 100% is unrealistic, as you would hire a personal assistant to do these kinds of tasks, and the personal assistant can be scammed, blackmailed, make stupid mistakes, or even be a foreign double agent. The problem is that, right now, AI models have more like the level of world-knowledge of a toddler, and so it is absolutely trivial to give them confusing instructions that they happily believe without much question.

But like, let's say you wanted to hire random, minimum wage level gig economy workers (or you wanted to leave your nephew in charge of the store for a moment while you handle something) to manage your mail... what would you do to make that not a completely insane thing to do? If it sounds too scary to do even that with your data, realize people do this all the time with user data and customer support engineers ;P.

For one, you shouldn't allow an agent--including a human!!--to just delete things permanently without a trace: they only get to move stuff to a recycle bin. Maybe they also only get to queue outgoing emails that you later can (very quickly!) approve, unless the recipient is on a known-safe contact list. Maybe you also limit the amount or kind of mail that the agent can look at, and keep an audit log of all of the search queries it accessed. You can't trust a human 100%, and you really really need to model the AI as more similar to a human than a software algorithm, with respect to trust and security behaviors.

Of course, with an AI, you can't hold anyone accountable really; but like, frankly, we set ourselves up often such that the maximum level of accountability we can assign to random humans is pretty low, regardless. The reason people can buy "unlock codes" for their cell phones is because of unaligned agents working in call centers that lie in their reports, claiming the customer that merely called asking a silly question--or who merely needed to reboot their phone--in fact asked for an unlock code for a cell phone (or other similar scam).

3 comments

AI is also scalable: if I find a text-based way to mind-break your minimum wage email sorter, I get one inbox. If I find a way to mindbreak apples llm email sorter, I get 30 million inboxes. In addition, I can try a thousand times to work out how to trick the email sorter on my account, and then transfer that solution to Robert CFO’s account if I suspect he uses the same model
It is not as easy as that. AI responses currently are probabilistic (intentionally).
"... as you would hire a personal assistant to do these kinds of tasks, and the personal assistant can be scammed"

Which is why I've never hired a human assistant and given them full access to my email, despite desperately needing help getting on top of all of that stuff!

Most people should be very uncomfortable giving a random gig economy worker access to their personal email accounts that act as fallback authentication for everything in their lives.

The fact that our AI systems have this level of trustworthiness is a big problem for harnessing their potential, since you want them to be a lot more trustworthy.

But AI is even worse, it has no sense for when things are weird and it is under attack. If you sent a hundred messages to a human trying slight variations of tricks on them, they would know something was wrong and they were under attack, but an AI would not.