You don't have to use the hottest new framework or language feature. The JavaScript you're writing now will still be supported while the language evolves and improves.
Many libraries that use ES6 features distribute the transpiled version through npm or GitHub. If they don't, you can just transpile the library using Webpack then use that.
If it was just about transpiling, that would be fine. The problem is that some frameworks (eg: React) add on a whole lot more assumptions about how how your app works. Just look at the plethora of blog posts, tutorials and even whole mini-frameworks just to get d3 to work with React.