Hacker News new | ask | show | jobs
by mewpmewp2 763 days ago
It's impossible to evaluate the author though since we didn't get any information. For all I know the high level contacts also didn't have time to concentrate and just told to "report it".

But there are strong and confident claims from the author "What do I do with this powerful and potentially dangerous prompt".

So is the output or result of this prompt anything more than what you would get from high temperature settings?

It's hard for me to think of a security vulnerability here.

Also the example links about prompt attacks etc are about getting past an LLMs censorship. I don't think these are security flaws. The reason why LLMs like these have censorship is just basic PR. I don't think it's a huge issue if it's possible to bypass and ask steps to do something illegal. If anyone seriously wants to do that, they will find a way outside the ChatBot anyway.

Anyone willing to go to Darkweb to get this prompt attack and then use the prompt attack in the actual LLM itself where it might get logged would only put them at risk of getting caught compared to if they just read the instructions on how to do illegal things on Darkweb.