[mindcrime]

I would say that if you're using a single layer NN, the answer is "you don't really know". And that gets to a point about how we still don't entirely understand how neural networks work, even when they do work.

If you were using a deep network though, and if the current theory is correct, it would be a slightly different story. The current thinking, as I understand it, is that with deep networks, each layer learns representations of certain features (say "slashes", "edges", "right slanted lines", "left slanted lines", etc.) and the progressively higher layers learn representations composed from those more primitive features. So if a deep net were recognizing your handwritten characters, you could probably reason that it isn't just considering whether the number of black pixels is even or odd.

Now in reality this is a pretty contrived, and probably unlikely scenario. But it's a valid question, because there's a deeper point to all of this, which involves transference of learning. That is, how do you take the learning done by a neural network - trained to do one thing - and then leverage that learning in another application. We still don't exactly know how to do that, and that's in part because we don't entirely understand the nature of the representations the networks build up. So a very good answer to your question would arguably help understand how to do transference, which would make NN's even more useful.

[argonaut]

The net is just going to learn the representation "even/odd number of black pixels," if that's the easiest thing to learn.

It also goes without saying that 2k images is probably not going to be enough data to learn any meaningfully general feature representation.