[Jarwain]

I mean it's not going to invent new words no, but it can figure out new sentences or paragraphs, even ones it hasn't seen before, if it's highly likely based on its training and context. Those new sentences and paragraphs may describe new ideas, though!

[sneak]

LLMs are absolutely capable of inventing new words, just as they are capable of writing code that they have never seen in their training data.