Have you considered using Pixi.js instead of pure canvas? Pixi.js uses WebGL with a canvas fallback and performance is much better:
http://www.pixijs.com/
Early on I took a close look at Pixi.js, though it turned out to not be the right fit for the project. However, it is planned for Yeah to have an interchangeable WebGL renderer at some point, similar to Pixi.js and Three.js.
If you're looking at WebGL, check out StackGL. It's pretty much intended as a low level WebGL wrapper with a modular codebase, and lots of high level modules built on top of it.