| >The requirement is to prevent cross talk entirely, Quite frankly, no it isn't. Interacting signals can be fully recovered. You can lose information by combining information, but it doesn't necessarily have to be the case. >The model you describe is not an LLM But this is a claim you can also make of any proposal that might fix the problem of prompt injection, but if you admit that it does solve the problem then to claim that your definition of a LLM must be vulnerable to prompt injection relies on one of the differences between these two architectures. It's easy enough to imagine a model with a similar command stream and input stream each with their own attention mechanisms and a cross attention between them. You can call it not an LLM but then your have a stricter definition that is not interesting. You end up claiming like a broken car will never drive because if you fix it it isn't a broken car. True but not worth claiming. So far the arguments are that once you multiply unknown values by parameters and sum them you cannot retire the original information. So that if your input is a and b. And you go through a layer of weighted multiplacation and addition the values are hopelessly intertwined. So if the layer had weights of c,d,e,f, you'd end up with P=ac+bd and Q=ae+bf. And both values contain a and b, is that correct? But since the model contains the weights c,d,e,f it could also learn a weight of Z= 1/(cf - de). It's just another constant after all.
And if it in a following layer it had weights of f,-d, c -e
Then it would produce two outputs of A=Pf + Q-d and B=P-e + Qc A and B are proportional to a and b. Multiply them by Z to get the original values back. Combining is not the same thing as signal loss. |