[lopmotr]

Who's going around designing a system that will have severe consequences without mitigating the problem of misclassification? There are techniques available, such as autoencoders, sensor fusion, ensembles, using multiple images, training on "neither" examples, asking for human confirmation, etc. It might never be perfect but neither are humans. We see monsters in the dark, the virgin Mary on toast, a face on Mars, optical illusions, get our attention distracted (magic tricks), act maliciously, fall asleep, etc.

The weird failure modes thing already happened with lossy image compression. Characters in non-OCRed text go replaced with different ones by photocopiers, and people saw spaceships in space probe photos of the sun. We'll get used to the odd banana riding a motorbike and realize what's up.