[manthideaal]

I wonder if a two step process could work better than this, first use a variational autoencoder or simple an autoencoder then use it to train the labeled sampled.

In (1) there is a full example of using the two step strategy but using more labeled data to obtain 92% of accuracy. Someone can try changing the second part to use only ten labels for the classifying part and share results?

(1) https://www.datacamp.com/community/tutorials/autoencoder-cla...

Edited: I found a deep analysis in (2), in short for CIFAR 10 the VAE semi-supervised learning approach provides poor results, but the author has not used augmentation!

(2) http://bjlkeng.github.io/posts/semi-supervised-learning-with...

[amitness]

Yeah, authors have tried mixing the strategy you described(self-supervised learning) for semi-supervised tasks.

Basic idea is to learn generic image representations without manual labeling and then finetune that on your small dataset. These are relevant articles I have wrote on it: https://amitness.com/2020/02/illustrated-self-supervised-lea...

https://amitness.com/2020/03/illustrated-simclr/