Hacker News new | ask | show | jobs
by cloudking 605 days ago
OP is demonstrating that the product follows prompts from the pages it visits, not just from it's owner in the UI that controls it.

To be fair, this is a beta product and is likely ridden with bugs. I think OP is trying to make a point that LLM powered applications can be potentially tricked into behaving in ways that are unintended, and the "bug fixes" may be a constant catch up game for developers fighting an infinite pool of edge cases.

2 comments

Saying 'tricked' is understating it. The example is Claude following instructions from a plain sentence in the web page content. There's no trickery at all, just a tool that's fundamentally unsuited for purpose.
For an LLM to read a screen, it has to be provided the screen as part of its prompt, and it will be vulnerable to prompt injections if any part of that screen contains untrusted data.