Yes, they're implemented with Polymer, but that does not mean they're "Polymer specific". You can use them in any HTML page - just import the element, and use it. This is the promise of web components.
Material design is more than just the colors, shadows and spacing, its also the animations and interactions.
This CSS framework fails to replicate things like "ink" that fills an element with a tap color based on where in that element you tapped.
Or the way that a block of color slides across the input when you focus it and the placeholder slides up and scales down to turn into a label.
Material design is not a CSS theme, it requires many other interaction and animation aspects that the Polymer project is building out with their paper-* components.
I'm aware of that, all I was saying is that hypothetically it would be possible to dismember the polymer components in separate pieces of javascript / css / html and replicate the behaviour in this way. I'm not sure why someone would want to do that, but just studying the implementation might be interesting to someone new. The paper-components seem very well designed.
In any case it does make way more sense than trying to replicate the design on your own and reinvent the wheel. Especially since there is nothing particularly creative in copying the design of someone else.
Their implementation does not only consist of CSS - there is canvas, shadow DOM, and other areas that are all heavily intertwined with Polymer, which would make extracting it difficult and complex.