[schrep]

This is some of the most advanced work out there - but CV is not “solved” most vision systems only can label about 1k categories of objects. So capatchas can still be easiy constructed that would fool these systems. Part of why it is exciting to get this out there others can help us improve it.

[dx034]

But how many more classes can all users solve? If you have to identify all oaks from a group of trees I doubt many would solve that correctly.

[m_ke]

Imagenet is the only reason why most models are trained on 1K of categories. There's plenty of models in the wild that handle 10s of thousands of classes.

[schrep]

With what precision and recall?

[m_ke]

Not that much worse than what you see on imagenet. Most large companies have internal datasets with >100 million images.

https://arxiv.org/abs/1610.02357

[schrep]

That’s impressive work. Still don’t think we have reached human level for all the categories of things we see in images. But you are correct that my comment about 1k categories is not true for many production systems.

[jph00]

Whilst there are plenty of things in CV that computers aren't super-human at yet, object classification (given 100+ examples) is not one of them. In datasets with tens of thousands of categories, humans are much worse than computers - e.g. humans are really not good at knowing the difference between every type of mushroom, algae, and model of airplane.

Further, nearly every time a computer recently has been trained to do some very nuanced classification, such as in radiology, they exceed human expert performance.

(Outside of classification, computers are rapidly making progress - for instance they are getting surprisingly good at predicting the next few frames of a video, which requires a lot of "world knowledge" to do correctly.)

[m_ke]

Definitely not close to having things work for all categories. As you scale up to more categories ambiguity and specificity becomes an issue. Clarifai has a nice demo of their model which has >10K classes, https://clarifai.com/demo , the top predictions are usually correct but not always the most relevant.

I only linked to the xception paper because it mentions JFT. It's not state of the art for large scale recognition.

[visarga]

It's not just a matter of detecting objects and locating them. The deeper computer vision problem is to identify object attributes, relations between objects and actions in video. It's much harder to do that because many relations appear in very diverse situations, with objects of different categories, so it's hard to have 1000's of examples for each class of relation.

For example, humans can identify a monkey riding a Segway on the airport runway, but there probably is no such thing in the training set, even if it is quite large. The neural net might not know if that constitutes a "riding" action because it has never seen such a combination. Maybe the monkey is jumping over the thing and the picture shows it in proximity to it, not riding it - a human would know that a slight gap means there is no riding taking place.

Then, the even harder problem is to predict the consequences of actions on objects and just to physically simulate the scene. Such knowledge is useful in robot action planning. Beyond computer vision, there is also a need to create a "mental simulator" that has theory of mind and can simulate other agents (what humans intend), and we need simulators, both physical and mental to create the next level of AI.

[humanbot]

Interesting. Can you teach me what is the state of the art for large scale recognition? Would like to read more about it. Thank you.

[adrianN]

You don't need human level to make CAPTCHAs useless. If you can break the CAPTCHA 10% of the time that's already enough.

[itsa2]

In case you didn't realize, the guy you're talking to is the CTO of what some may call a "large company."

[schrep]

Doesn’t mean I’m right :) - see above as he had a point.

[mcintyre1994]

Assuming from context you're CTO of Facebook, the facebook.com/schrep link in your profile isn't working:

This page isn't available. The link you followed may be broken, or the page may have been removed.

[tvmalsv]

And I enjoyed the conversation between two people that obviously know the field pretty well.

[m_ke]

Wow that's a great catch.