[jph00]

BTW, one thing which makes it a bit hard to get into self-supervised learning is that the most common benchmarking task involves pretraining on Imagenet, which is too slow and expensive for development.

I recently created a little dataset that is specifically designed to allow for testing out self-supervised techniques, called Image网 ("Imagewang"). I'd love to see some folks try it out, and submit strong baselines to the leaderboard: https://github.com/fastai/imagenette#image%E7%BD%91

[allovernow]

I might take you up on that. How does your dataset facilitate self supervised experimentation?

I've a good amount of experience playing with autoencoders but this is the first I've heard of contrastive learning.