Hacker News new | ask | show | jobs
by DiscourseFan 677 days ago
An LLM takes a slice of data from the world, by nature it has to organize it in some such way, depending on how its trained, and the method of organizing it is hard-coded into the model. Therefore, all models will develop some sort of style, no matter what, since somebody, or a team of people, had to figure out a way to portion out a selection of data, and this problem is intractable.
1 comments

generative models are trained to generate outputs in response to an input, that closely resemble the training data. that’s literally all they do. if a base model was introducing “style” training (as we currently do it) wouldn’t even function. what you’re implying is mathematically intractable for generative models, and that’s fundamental to what they are and how they are made. the style stuff you’re referring to is a side effect of fine tuning and contexts of chatbots, it’s not a property of llms or generative models
So you agree with me? Style is fundamentally part of the set of all data used in production, and that can be “tuned” as you say, but never removed. Its the ghost in the machine, the spark of contingency. Of course, all machines bear the mark of their creators, but LLMs doubly so, as creators themselves. Like shitty, partially incoherent children.
the models used in OP site are not tuned on stylized content

you keep saying LLM when you mean chatbot, i’m not sure if you’re really reading my posts