[daxfohl]

Could you use LoRA adapters to free up your context with all the stuff that normally has to go into it? Coding standards and fuzzy preferences like "prefer short names" or "prefer functional style", reference materials, MCP definitions, etc.?

For training data, I was thinking you could just put all the stuff into context, then give it some prompts, and see how the responses differ over the baseline context. You could feed that into the fine tuner either as raw prompt and the output from the full-context model, or as like input="refactor {output from base model}", output="{output from full-context model}".

My understanding is that LoRA are composable, so in theory MCPs could be deployed as LoRA adapters. Then toggling on and off would not require any context changes. You just enable or disable the LoRA adapter in the model itself. Seems like this would help with context poisoning too.