[fuzztester]

why are there so many parameters in the first place? and was it humans who generated so many? seems like a very big job for a human to do, or even a team of humans to do.

disclaimer: I know next to nothing about llms. and I'm not that interested to learn about them. just asking casually.

[Jensson]

> why are there so many parameters in the first place?

Because parsing and writing human language in a natural way is extremely complex.

> and was it humans who generated so many?

No, it is generated using an algorithm that tries to predict the next word in human written text using the words that comes before it. It ingests basically all the text on the internet to do this, without that much text the LLM performs horribly.

[jdiff]

They're not manually generated or anything, it's just a setting. Too few and the model doesn't have enough flexibility to capture complex patterns. Too many and the model can just memorize the data you train it on rather than capturing the patterns driving it.