[bradfox2]

Having done this for domain specific engineering paperwork that looks similar to cause analysis, it does work well at param sizes << 70B.

lora does not work though, you need full parameter training if the knowledge isnt already present in pre training set.