[lukasco]

This is the real problem with LLMs. There is no way to separate code from data. At best, models could be trained on tokens that indicate untrusted data coming in. But then the untrusted tokens could also be messed with.

I've wondered if it would be possible for there to be two input streams: 1, for prompt, 2 for untrusted data. But I suspect that transformers would still only optionally decide what each one was for. So it would still be a prompt level suggestion, rather than a hard and fast rule.

[moron4hire]

LLMs should never be trained on restricted data of any kind, as we have seen that they are able to reconstruct their training data. The idea that they could be trained on private/restricted/copyrighted data and that was ok because there wouldn't be redistributing that data should have been killed 3 years ago.

Embedding vector indexes are how we separate code from data. Anything that is not for 100% unadulterated public access should be behind a traditional access control system. RAG search is not magic, it's just a SQL query of a manually created index. It absolutely could have access control built in. It's been out of laziness that it has not.

[zephen]

I cannot disagree, but many who should know better do.

I have seen people argue with a straight face that there are no copyright concerns simply because of the sheer volume of the data that LLMs are trained on.

This makes less than zero sense. If someone has seen code, or heard music, and creates something too similar, it is a copyright violation, even though that person has seen much code or heard much music before. This is why the concept of "clean room" implementation exists, and why the concept of the abstraction-filtration-comparison legal text exists.

LLM proponents will point to the fact that courts have ruled that using copyrighted material for training has been ruled fair use.

This actually makes sense. Just as you can read a book, so can an LLM.

The thing that, AFAIK, hasn't been ruled on yet, is when the LLM regurgitates something that is too close to the book. If a human were to do that it is a clear copyright violation.

To pretend that "dilution is the solution to pollution" in terms of LLM training data, and that anything the LLM produces is original material, is to give LLMs more rights than humans have.

[Terr_]

> The idea that they could be trained on private/restricted/copyrighted data and that was ok because there wouldn't be redistributing that data should have been killed 3 years ago.

I know, it's kind of mind-boggling. It's not old enough to be a history problem [0], and it's hardly a secret, so I have to assume it's a state of denial.

In other words, there are groups of developers and executives (and investor-money) with a collective motiviation to desperately shut their eyes and pray that Bad Thing simply won't happen to their product, at least not before someone else invents a revolutionary Silver Bullet that makes it go away.

[0] "Those who cannot remember the past are condemned to repeat it" -- George Santanya

[lukasco]

I feel like in some ways this problem is starting to self-correct, sadly by creating the dead internet. If there's no business model to creating content since it will get scraped, then no content will get created.

[sandeepkd]

My perception of real problem is that the LLMs were generic purpose tool and the focus was to improve their information retrieval and prediction. And they were fed with all this data (including private with was otherwise not available to everyone) for training purposes. The security and privacy of stored information was not really the requirement of this whole endeavor and all of sudden in the real world they are finding that this is a must requirement if they want to sell these models to enterprise companies.

And now all these security efforts to manage data privacy are akin to lipstick on a pig, they are brittle, costly, one-off. There are no boundaries inside the LLM storage, the training data is not encrypted at all in the memory across the pseudo tenants

[jerjerjer]

The joy of nondeterministic systems.

[sandeepkd]

The LLMs are beautiful if anything, a lot of creative and hard work has been poured into building them. They take the natural language to the next level and all that was engineering part. It has its own usefulness subjective to areas. The business part, trying to put it as a silver bullet for everything, is trying to put a square peg in round hole is the one which is causing this polarization.

[blourvim]

You could have limited-instruction llms where the model does one thing, for example summaries. It could accept a limited amount of instructions for example, first token for verbosity, second for style etc...