[axelroze]

It's not that simple because that way you lose too much information. Actually it is more likely the whole system would fail if weights at any layer are reset.

There is a way to selectively unlearn something via Memory Aware Synapses (MAS): - https://arxiv.org/abs/1711.09601

The idea was developed mostly for transfer learning as in learn new stuff on a new domain but do not forget the old stuff as well. For forgetting it could be trained on some old images + all zeros target mask and the MAS to preserve everything else.

[odnes]

Both MAS and (the earlier) EWC facilitate continual learning through passing a bunch of samples through the network and collecting gradients to determine which weights are 'important'. Future weight changes are then regularised by these importance values so that the network retains its ability on past tasks. EWC uses square gradients as importance values, whereas MAS uses absolute gradients... Other than that they're the same lol (I think), how the MAS paper got so many citations I have no idea.