That's a gross oversimplification: Flash and Silverlight were rich platforms with complexity on the same order as the entire browser. EME has a much narrower interface which provides stream decryption – it doesn't even have the video codec, whereas Flash/Silverlight had complex video, audio, image, PDF, font, etc. implementations with a long history of exploits.
This really matters because so many of those exploits relied on other features to actually run the payload. Not having any of those in the first place is a big attack surface reduction, even if the politics are legitimately debatable.