The problem is that idiomatic JS and blazing fast JS are diametrically opposed to each other, in practice the latter is more like a bad C dialect. You're not allowed to allocate GC objects in fast JS but the language doesn't have good non-allocating alternatives. Nobody is actually going to make a complex JS app where all memory allocations are pointers into a giant ArrayBuffer, it's easier to just switch to WebAssembly at that point.
If JS had typed structs (like they have type arrays) it would definitely be more convenient.
However, that's not where the problem starts. A lot of web sites are slow because they simply run too much code that doesn't need running in the first place and allocates objects that don't need to be allocated.
We don't need lower level constructs if we can simply start by removing cruft and be more wary of adding it. Go back to KISS/YAGNI.
"Too bad we cant just rely on JS only and have to involve a bunch of DOM operations, which is usually the slow part of the UIs we create"
No? With WebGL and soon WebGPU, or in this case here with writing to a imagebuffer and just passing that to canvas, you don't have to use the DOM anymore since quite a while.
(but then you don't get all the nice things html offers, like displaying and styling text etc)