[kaffekaka]

I don't think it is that simple. Innovation can be found on several levels, from the lowest next-token level up to the higher level of new ways of combining things. Surely LLMs can produce code that on the whole does something completely new, even if on a syntactical level it has all been seen before?

We only have a couple dozen letters, still it is possible to write new poetry.