[danw1979]

Yes, neural networks are famously poor at generalising.

[macleginn]

They are poor at generalising from a small number of examples; this is why the real generalisation power is achieved in pre-training.