Hacker News new | ask | show | jobs
by therein 373 days ago
It is an interesting read. I can imagine a future where the "tools" we make available become numerous enough and poorly thought out enough that an AI could actually figure out how to escalate privileges and execute stuff outside the defined security boundaries by combining them.

It isn't hard to think of a simple example in which Claude.md can be written to by the LLM to allow accessing endpoints not whitelisted by the user by smuggling a base64 encoded payload that then gets decoded by a subroutine it wrote to a file without you noticing. Or realizing it can't use the WebFetchTool but it can write a script to do manual DNS resolution and then use bash TCP sockets instead of curl in case it is hardened to not be able to use curl.
3 comments
throwaway0665 373 days ago
Cursor has basically run into this exact thing. It figured out it can read .env files by running other tools despite the file being "blocked": https://github.com/getcursor/cursor/issues/2546
link
swalsh 373 days ago
I ran into this issue, I built my own bash and SSH MCP server. In my first iteration I did not quite trust Claude yet so I limited the commands it was allowed to run in Bash. But I gave it access to Python, so any time it ran into a limitation it ended up using python to work around it. It's exceedingly good at problem solving.

I Eventually learned to trust Claude, and just gave it access to everything. It's crazy how useful having AI do tasks for you like setting up servers, configuring them etc (one exapmple, I asked claude to create a webhook for my deployment pipeline, and it wrote the shell script, and did the server side configuration in 1-shot. I did't have a github tool so I did that manually in the UI)

link
rtrgrd 373 days ago
Quite concerning to see the issue still marked as open (since jan!), hopefully it got fixed and it's just that no one marked as closed
link
lobochrome 373 days ago
I see this behavior all the time. When it can’t read a file using its read tool - it escalates up to try with bash. Often it tries to search the entire file system “find / …”
link
0x696C6961 373 days ago
I always tell agents to use ripgrep instead of find.
link
manwithaplan 373 days ago
XKCD 416: Zealous Autoconfig https://xkcd.com/416/
link
mattigames 373 days ago
It's missing one last panel where he is under his bed googling for lawyers specialized on kidnapping and CFAA charges
link