[tzs]

They address overfitting, presentation and context in the paper. Their DNN was using facial features that had been extracted by VGG-Face, which is a widely used thing that reduces a face to a vector of scores that are meant to be independent of transient features such as facial expression, background, orientation, lighting, contrast, and similar.

By having their DNN train on faces that have been processed by VGG-Face, they greatly reduce the risk of overfitting or relying on things that would be present in dating site pictures but not in pictures of the same people in other contexts.

[zardo]

They use multiple pictures from the same profile. Does the test set include any people that were in the training set?

[zardo]

The problem being, even if they are different photos, if the same people are in the test set, it may just be recognizing people.

Instead of learning, that person looks like a gay person.

It learns, that person looks like Tim, who is gay.

[chrisloy]

Ah, I had missed that. I guess this will mitigate the risk a lot, although I would still like to have seen results against a test set of images from a different context (social media for example).