[m3at]

Yes this is more often called self-supervised.

Note that most sample pairings, especially for images, is done through augmentations currently, so the implicit labeling you're doing is still weak on priors.

Of the methods mentioned in the article, BYOL (and even more the follow-up SimSiam [1]), have the weakest assumptions and work surprisingly well despite their simplicity.

[1] https://arxiv.org/abs/2011.10566

[zwaps]

I agree with Op that this is still essentially learning on labeled data.

I say this, since there are also cases of constrastive sampling like ideas with truly unsupervised data. For example, Graph Embedding, where a graph implies structural features of similarity and distance that the representations should capture.