[mkrum]

I remember a story a professor once told me about something similar to this. He once knew of a research team that was trying to develop an algorithm to tell the difference between pictures of American and Russian tanks. They were able to achieve a very high success rate very quickly. Excited but skeptical, they decided to keep testing the algorithm on lower and lower resolution photos. Shockingly, they were still getting close to 100% identification on images of sizes around 10 by 10.

Turns out, all the pictures of Russian tanks were taken in the winter, while all the american ones were taken in the summer. All they had done was trained a model to classify how bright the picture was.

[gwern]

Yeah, this urban legend always gets trotted out to criticize neural networks, but after years of looking, I've never been able to confirm it, and even when Minsky tells it, he can't name any names or concrete details about when or where - for example, in Minsky's version, it was how the photographs were developed, but in yours, it's winter/summer, and in other versions, it's night vs day, or it was forest vs grass: http://lesswrong.com/lw/td/magical_categories/4v4a http://lesswrong.com/lw/lvh/examples_of_ais_behaving_badly/c...

[mindviews]

And last week I heard that story but the difference was that the images of the NATO tanks were in perfect focus while the Soviet tanks were out of focus in the training images. Clearly this story is all over the place. I wonder if even the likely source is correct. https://www.webofstories.com/play/marvin.minsky/122