Have you used any other package manager? I'm not completely sure if it's just Node's dependency hell, but npm feels so incredibly slow compared to Cargo or bundler or something.
Never profiled it but I would guess that is largely due to the npm's main problem: It doesn't locked dependency versions. As a result it probably builds the dependency graph a lot more than it should.
For me this is the main thing I'd like to see fixed. Not only does it build more than it need to, it can result in broken builds. When my dependency doesn't update, I don't expect its dependencies to update (especially when I don't have control over setting the versions of the dependencies of dependencies). This has bitten me more than once.