[bigyikes]

Sure, doing surgery on the weights is not feasible, but doing surgery on the training data is possible (and is done for certain benchmarks).

That's where it gets interesting. If you tell the LLM what the bad data is, it could examine its own training set and fix the mistakes. Get enough LLMs with diverse enough training sets, and they will correct each other.

If models are stochastic parrots, you might expect training data to devolve into corruption via a negative feedback loop, but the opposite might happen if models are actually intelligent.