[extasia]

>It is counter-intuitive that LLMs can exhibit such resilience despite severe disruption to input tokenization caused by scrambled text.

I'm not sure that i agree. an LLM maximising the likelihood of its output could surely permute its input in such a way that it unscrambles the text?

Need to read a little deeper and will report back.

edit: interesting result, but the paper doesn't present a good reason that this would be "counter-intuitive" imo.

[spuz]

One way to consider how an LLM "sees" text is to imagine a character based language like Chinese - each symbol is a syllable which can be a word on its own or part of a word. If you scramble words at the character level, you are going to produce combinations of letters that don't match any known symbol. It would be like drawing a series of random strokes and asking someone who knows Chinese what it means.

If you look at the example given in the paper, the word "won" is a single token. When it is scrambled as "wno" it is tokenised as "w" and "no" both of which are unrelated to the original token "won". Somehow the LLM is able to relate these two completely different tokens "w" and "no" back to the original token "won". I think the paper is claiming this is surprising because these tokens shouldn't have any correlation with each other in its training data.

[evertedsphere]

Partial counterpoint: if you were to mutate characters (by changing less distinctive components to other vaguely similar-looking ones) or to reorder the order of characters in hanzi/kanji compounds, I would imagine that a native speaker would still be able to read them with some difficulty.

I can attempt to produce a Japanese example by going to town on an example from Jreibun, but note that I am far from native:

後食に罠くなるのは生里像現なので壁けることはできないが、午後の事士の校率が干がるので木っている。

As far as I'm concerned, swapping out the radicals doesn't hurt that much (this is usually a negative, since it leads you to confuse character pairs like 候 and 侯, especially if you don't practice writing) and swapping the order of characters is a bit more annoying.

That said, a Mandarin one would be more convincing, since reordering the various markers that serve the roles of Japanese verb conjugations would be less disruptive than turning できなかった into っぎかてなた, which I did not do for that reason.

--

(The original sentence was 食後に眠くなるのは生理現象なので避けることはできないが、午後の仕事の効率が下がるので困っている。)

[xondono]

Like GP, I’m not sure I find it very surprising.

Presumably chatGPT has not only lots of typos in it’s training data, but also even nearly scrambled text with things like Pig Latin.

In a sense being able to process scrambled text is an overpowered version of typo tolerance.

[thaumasiotes]

> If you scramble words at the character level, you are going to produce combinations of letters that don't match any known symbol. It would be like drawing a series of random strokes and asking someone who knows Chinese what it means.

You can see an example of doing that very thing here: https://pbs.twimg.com/media/EG1ZV_tX4AALiIG?format=jpg&name=... . There is a hashtag at the bottom of the image explaining the meaning of the nonexistent character, but if you remove that from the image, people understand it just as quickly.