How far can you get with JS/other interpreted things in e.g. optimizing for cache access etc? Sounds like you're at the mercy of the JIT compiler (which may go far, but still).
The interesting question to ask is whether these Rust rewrites are really taking advantage of cache optimization or if they are making simplifying assumptions that the canonical implementation cannot. In the latter case, Rust isn't the root of the performance difference and a JS rewrite can make most of those simplifying assumptions
JavaScript does not have any input/output (IO/syscalls) all those functions like reading a file, socket, etc needs to be implemented in the runtime language, like the browser, Node.JS, ASP, etc. So you are at the mercy of the runtime executable. The JS JIT slows down startup time as the JavaScript first have to be parsed and compiled before running, but the runtime can then cache the compiled version for faster startups. When JavaScript was new it was slow, for example string concatenation was slow, but most JS engines now implement string concatenation with a "rope" structure making it very fast. v8 (Chrome) and Spidermonkey (Firefox) have got a lot of optimizations over the years, so if you are doing things like incrementing a number in a loop it will be just as optimized as something written in a systems level language.
This can vary a ton as well. I was talking to a friend yesterday who said he was pounding a native browser interface with an iterator and experiencing slow performance. He switched to buffering the entire thing into memory first and experienced huge performance gains.
The aspect of the language you're using, if optimized, is virtually always optimized for the most common use-case. If your use-case isn't the most common use-case, you must account for this.
With stuff like TypedArrays pretty far. Where JS has problems is optimising in the face of the things you can do with the language and the current difficulty in multithreaded implementations.