[colincsl]

You're right that the video looks very impressive. However, I'm hesitant to believe that this actually works as well as one might perceive. In this area (called semantic segmentation in the computer vision world) it is common for a people to highlight both the good cases (where the labels/segmentation are correct) and the bad cases (where they are incorrect). In the MS video they only show the good. It's easy to cherry-pick examples where it works -- even if the overall accuracy is very low.

Furthermore, I don't know which dataset they're using. Perhaps it only works on a small set of objects such as those shown in the video.

I don't mean to knock their results but it will still take time to get this to work on a more broad set of videos.