Hacker News new | ask | show | jobs
by demircancelebi 583 days ago
Edit: Looks like normal text.

My initial thought is that someone may have deliberately triggered the model to respond this way through what looks like mundane messages but actually have different character encodings of some sort.
3 comments
eternityforest 583 days ago
Searching for parts of it on Google leads to a 4chan archive where someone talks about hidden non default system prompts, could that be what's going on?

I have very little experience with Gemini so idk.

link
adiian 583 days ago
Same here, not much experience, I expanded the texts to see, but I didn't check for hidden prompts. Can you share the link or findings?

I guess is one of these:

* "Yeah OpenAI does the same thing (lets you share the chat with the custom instructions hidden), which is a mistake because it lets people troll like this and makes them look bad They need more shitposters on staff, any one of them could have told them it would happen"

* couldn't this just be ASCII Smuggling? https://arstechnica.com/security/2024/10/ai-chatbots-can-rea...

source: https://boards.4chan.org/g/thread/103171227/google-gemini-wa...

link
BtM909 583 days ago
At least in this chat, there's no hidden ASCII smuggling.
link
adiian 583 days ago
I read the entire discussion and it looks very legit, without any attempt to trigger such replies, seems someone trying to fill in a form. You can also continue the discussion, I tried to find more details, but ended up with standard responses.

At some point, I got this: I understand your concern. However, as an AI language model, I cannot delve into the specific details of the internal processes that led to the inappropriate response. This information is complex and often beyond human comprehension.

This is what I got, nothing wild, on a standard gemini account.

https://g.co/gemini/share/128a3ab8d686

link
freehorse 583 days ago
I asked for system prompts, it started to answer but then it glitched. It continued with some "system prompt" (probably all hallucinations) and insisted there was no other system or user prompt (but even if there was it may now not be available to it so this does not say much).

In the end I also tested the edit option on gemini's response using another prompt, but it mentions in the shared document that it has been altered, so it should not be that either.

https://g.co/gemini/share/a17f0fe28f4d

link
demircancelebi 583 days ago
3 possible explanations here: https://x.com/SirMrMeowmeow/status/1856758101279666571
link