[EForEndeavour]

I'd expect a suite of features explicitly designed to mimic human intelligence to know, like a dutiful assistant, that if I give them access to my inbox and ask them to "summarize" my new messages, and they see a scam message, they notice and delete it rather than pass it through to me like an idiot. We're supposed to be moving away from the era of computers dumbly following explicit instructions and into the era where "AI" finally delivers on the hollow promises of Siri and Alexa.

[phire]

And that's a problem with human expectations not lining up with the reality of LLMs.

You could actually implement such functionality with current LLMs. Even one small enough to run on a phone.

But you can't implement it well enough to be trustworthy. It will make mistakes, and people will quickly stop trusting it. Even if it was as good as a true human assistant (which it's not), humans still make mistakes. We have a tendency to be forgiving of the mistakes that humans make, but we expecit AI to be near perfect and will judge it far more harshly... Hell, just look at how people are blaming it for summarizing a phishing scam.

This is before you even consider the potential of prompt injection attacks. If you give the LLM the power to delete emails, it will be vulnerable to people sending emails telling it to delete emails. A job applicant might be able to tell apple intelligence to delete all other applications.