[sanchezdev]

I basically agree with this rule. I find that my colleagues who overly hype unsupervised approaches typically don't have much experience working on ML problems without labeled data. My suspicion of this comes from the fact that whenever I give a talk on ML I always have a wealth of personal experience to draw on for examples. My colleagues almost always reuse slides from projects they never worked on.

[sillysaurusx]

I'm a little surprised to see this sentiment. Some of the most important advances in the field have been unsupervised tasks:

- OpenAI: Dota 2 (PPO), GPT-2...

- NVidia: StyleGAN, BigGAN, ProGAN...

[sanchezdev]

Those are certainly important advances, but they don't really apply to most business needs for AI or ML.

[MiroF]

I work in the industry on NLP tasks. Unsupervised learning has been behind the largest developments in the last decade in the field.

[sanchezdev]

I don't disagree with your point, but the unsupervised aspect of NLP typically isn't useful on its own. Usually it's a form of pre-training to help supervised models perform better with less data.

From Google in 2018:

"One of the biggest challenges in natural language processing (NLP) is the shortage of training data. Because NLP is a diversified field with many distinct tasks, most task-specific datasets contain only a few thousand or a few hundred thousand human-labeled training examples. However, modern deep learning-based NLP models see benefits from much larger amounts of data, improving when trained on millions, or billions, of annotated training examples. To help close this gap in data, researchers have developed a variety of techniques for training general purpose language representation models using the enormous amount of unannotated text on the web (known as pre-training). The pre-trained model can then be fine-tuned on small-data NLP tasks like question answering and sentiment analysis, resulting in substantial accuracy improvements compared to training on these datasets from scratch."

[MiroF]

As I said, I'm an NLP researcher and practitioner, so you don't need to quote this at me.

The unsupervised aspect is the engine driving all modern NLP advancements. Your comment suggests that it is incidental, which is far from the case. Yes, it is often ultimately then used for a downstream supervised task, but it wouldn't work at all without unsupervised training.

Indeed, one of the biggest applications of deep NLP in recent times, machine translation, is (somewhat arguably) entirely unsupervised.

[sanchezdev]

I didn't mean to make it sound incidental although I do see your point. Just wanted to chime in with how important having a labeled dataset is for a successful ML project.

[Der_Einzige]

Echo the other commentator. Unsupervised techniques are the only reason NLP works as well as it does.

[zamubafoo]

I would argue that GAN's by definition aren't unsupervised, they just aren't supervised by humans. Additionally, OpenAI's game stuff also has similar arguments against it.

[MiroF]

> I would argue that GAN's by definition aren't unsupervised

You can define the terms how you want - but in terms of how they're understood in both industry and academia, you are incorrect.

[hervature]

The discriminator definitely is supervised but the generator is unsupervised. I.e., it has no labels on its targets.

[sillysaurusx]

I'm not sure that's correct. The discriminator and the generator both learn to match a training set. You don't need to label the training set at all. You can just throw 70,000 aligned photos at it.

I think I see what you're saying, but that might be a different definition of "supervised". It seems impossible for one half of the same algorithm to be supervised and the other to be unsupervised. But I like your definition (if it was renamed to something else) because you're right that the discriminator is the only thing that pays attention to the training data, whereas the generator does not.