[sumodm]

Something along this lines is the real danger. People will understand common failure modes and assume they have understood its behavior for most scenarios. Unlike common deterministic and even some probabilistic systems, where behavior boundaries are well behaved, there could be discontinuities in 'rarer' seen parts of the boundary. And these 'rarer' parts need not be obvious to us humans, since few pixel changes might cause wrinkles.

*vocabulary use is for a broad stroke explanation.