My comment was meant to imply that if you're using JS, then you already care about other things _more_ than you care about performance.
Once you've decided that you do care about those other things, then getting the best performance you can is great and deeplearnjs is very useful.
But if what you care about most is performance, then using JS is not the way to go. I'm guessing deeplearnjs is an order of magnitude slower (at least) for training a modern convnet relative to cudnn or MKL. And I don't think it's currently possible to use multiple GPUs? Although you could imagine some crazy distributed asynchronous training running in browsers all over the world.