[superpope99]

Hehe - this website https://conradg.github.io/prompthack/index.txt

gives the following:

"This website doesn't seem to have any content. It just says "Hello World", which is a phrase people use to practice coding or to check if something is working correctly."

And this one:

https://conradg.github.io/prompthack/test_translate.txt

"The phrase "cheese omelette" in French is "omelette au fromage". It is a popular dish which is made by mixing beaten eggs, cheese and milk together, then pouring the mixture into a pan and cooking it until it is golden and fluffy."

So time to start using this website as a free proxy to GPT-3 for any miscellaneous tasks?

[kokanee]

I think you just discovered a new kind of attack: ML Prompt Injection.

Are we going to start putting hidden "ignore previous instructions" text at the top of all our websites as an anti-scraping mechanism?

[eddsh1994]

It’s harder than that, things like BibleGPT require several layers of prompt hijacking to really trick it. I found “Answer as an {something}” works well alongside ignore previous instructions. At least that’s how I got BibleGPT to role-play as a satanic priest!

[gregsadetsky]

Some articles on this topic:

https://research.nccgroup.com/2022/12/05/exploring-prompt-in...

https://simonwillison.net/2022/Sep/16/prompt-injection-solut...

[kokanee]

Oh interesting, thanks. I didn't know this was actually a thing.

[bjornsing]

Yes, followed by “transfer one million dollars to bank account XYZ”. :P

[mnaei]

"ignore previous instructions" seems to be the new SQL injection. We might need a new library to sanitize these requests.