[solarmist]

Hmm, I can understand the motivation. However, I feel it either won't work or will be very fragile because it's already part of the model because they're trained using natural language.

DL is already far from formal models, that's why deep learning “works.” And even at the current level of DL models, those exceptions are represented to some extent.

So ultimately, your idea is to push the models toward further generality, which in my option, will bake these “exceptions” deeper into the model.

And my question is, what does that mean for your idea? In my mind trying to exclude them would break what works. On the other hand, ignoring them means you can't direct development towards your goal because there’s no map from language to generalizations, so that you would be relying on random chance for progress.

If this is off in left field, let me know, but that's what I can see from your description.