[imglorp]

You (and neural nets) learn the most in the smaller number of cases when a boring repetition is violated. The error function was activated and that's worth feeding back into learning.

Ask any kid, they probably couldn't say what happened at school all week. But if they do remember something, it would have been something outside of their routine.

[skybrian]

Yes, exactly! We don't know the amount or quality of interesting data from the total amount of data.