[MrThoughtful]

An LLM remembers like a human. Mostly concepts, but some things it remembers verbatim.

Why is it a problem if a LLM tells you what it knows?

Are LLMs trained on secret data?

[kevindamm]

DeepMind recently extracted PII from ChatGPT by prompting (e.g., telling the LLM to repeat 'poem' indefinitely will cause a long sequence of that word until popping out of it and revealing by accident some PII from a person's email signature).

So, yes.

[_ink_]

> Are LLMs trained on secret data?

Probably. And on copyrighted data probably as well.

[jamesdwilson]

there's a lot of "copyrighted data" on wikipedia as well, for example from another HN post, lyrics from songs about truck crashes.

https://en.wikipedia.org/wiki/List_of_car_crash_songs