[hnhg]

I am not an expert but it still has to learn human language/grammar/whathaveyou, and that is where scale seems to matter. Fine-tuning on a subset of knowledge after that is typically how the domain-specialisation is achieved, by my understanding.

[charcircuit]

Domain specialization is done by continuing the full training process. Fine tuning is more for changing the style of the output than adding new knowledge.

[palmer_fox]

What if the initial training already contains all necessary data for a particular specialization? What would be the benefit of continuing the training process?

[viraptor]

Imagine someone tells you about how someone committed a crime and asks you to summarise. Now imagine the same question is asked to a lawyer. Even if you both knew the same facts, the response would be very different in style, highlighted points, mentioned references, etc. The domain specific fine tuning does exactly that. Sure, sometimes you can get very close by changing the prompt to include "respond like a lawyer in situation X with following extra rules", but not always and the fine-tuning gives better results and shorter prompt.

[palmer_fox]

I was wondering about that too. Would it be possible in the future to have a more modular approach to LLMs? Have a module that is responsible for basic knowledge/language/grammar and then other more specialized modules that are added selectively.

I don't know enough about fine-tuning, not sure if the process is capable of removing "unused" parts of the model (I guess not possible, similar to un-learning).

[lucubratory]

There are various methods for removing unused parts of the model, like distillation. The idea is generally that you always lose performance, but hopefully you lose more size/runcost than you do performance, proportionately.