[ipunchghosts]

Eero simmoncellis group showed that specific network modifications allow the networks to extrapolate beyond noise they were trained in.

NNs can certainly learn beyond what they were trained for as new architectures have demonstrated. See benchmarks for any if the dan hendryks imagenet corruption dataset benchmarks (imagenet c r and a). Data + architecture work together to enable.

I have work using neural networks trained on synthetic aperture sonar SAS imagery using language only. The SAS datsets are extremely hard to come by. The dataset I used was from parts if world I can assure you we're never in the training set. My approach exceeds what clip models can do zero shot wise. Therefore, the network classified imagery if a modality and environment 100 percent not guaranteed to be in the training data.