[teravor]

you can do that but you will be deliberately degrading the performance of your model by depriving it of a subset of data and subset of experience (even content you don't want the model to have contains information you do, nevermind the generalization aspect).

with coding & exploit development you likely cannot even decouple the two if you wanted to, and if you can it will almost certainly cripple coding. note that reverse engineering binaries contains vast amounts of data for the model to train on to become good at bit engineering (understanding compilers, assembly, cpus etc).

[NitpickLawyer]

For sure. Now that I think about it, it would be a nice research question to answer: how much can you de-couple a field that is close to another and purposefully removed from the training data. At least ICL should bring back a lot, and as you say signals about one might come from different places.