[_Marak_]

If you are tired of waiting for NPM, I'd suggest checking out the `ied` alternative: https://github.com/alexanderGugel/ied

To give some perspective, I'm dealing with systems that install npm packages at scale. NPM3 has had way too many show stopping bugs for us. The delays in installation time compound when dealing with thousands of packages, pretty much causing package installation to fail ( take forever ).

[STRML]

ied is great, and getting better.

Unfortunately, as detailed in a few npm3 threads, npm3 performance may forever be quite bad, due to a few ill-conceived features in npm's infancy.

Due to shrinkwraps and bundleDependencies, it's impossible for npm to know the full dependency tree until it has untarred every intermediate dependency [1]. In the future, it may be possible for a smarter server to cache this data and serve the full tree as part of the metadata, but that is a ways away.

There are some low-hanging fruit still in npm; for instance, the npm registry still doesn't gzip metadata for some unknown reason. Doing so could save as much as 80% on some larger metadata json, such as npm/npm itself [2].

1. https://github.com/npm/npm/issues/8826#issuecomment-15511591...

2. https://github.com/npm/npm-registry-couchapp/issues/224

[fibo]

Try to set

    npm set progress=false

it speeds up installation time. I am also approaching an alternative solution, to put dev deps in another package.json, then do something like

   npm install src/dev

where src/dev is a folder that contains a package.json with dev deps only when you are developing.

[joshschreuder]

v3.7.0 of npm [1] fixed the previously submitted performance problem with the progress bar [2].

Which is not to say you are incorrect (as often disabling logging to the console can speed up operations), but for the most part the bug has been fixed.

[1] https://github.com/npm/npm/releases/tag/v3.7.0

[2] https://twitter.com/gavinjoyce/status/691773956144119808

[STRML]

It has not been fixed: I did a series of PRs on this [1] and the most impactful of them was not merged, although a few smaller ones were.

@iarna got to the root cause (instead of simply throttling it as I had been doing) but it has not yet been merged into npmlog. [2]

1. https://github.com/npm/npm/issues/11283#issuecomment-1757522...

2. https://github.com/npm/npmlog/pull/28

[joshschreuder]

Fair enough, I'm not up with npm's development on a PR level, so I was just going by this line in the changelog:

"This has been patched to eliminate that churn, and our testing shows the progress bar as being eliminated as a source of slow down."

Apologies for being misleading.

[STRML]

Just to update you - looks like this has finally been released:

https://github.com/npm/npm/releases/tag/v3.8.3

[joshschreuder]

Great, thanks! I'll check it out.

Unfortunately I've not had much luck getting pnpm or ied running on Windows due to the symlinking stuff (though the installation portion is super quick), so hopefully the speed improvements here can help lessen the blow a bit.

[joshschreuder]

As an aside, why does npm check-in the node_modules folder into their repo?

I was under the impression this was generally considered bad practice, so it seemed weird seeing it done by the package manager itself :)

[untog]

While progress=false does decrease install time a little, it's still very long. And the output is incredibly verbose - I'm doing something similar to the OP in automating builds of Node apps and npm is easily the biggest headache in the process.

BTW, "npm install --production" installs all dependencies but not devDependencies (which you can get by doing "npm install --save-dev <package>"). That way you can separate dev and prod dependencies within the same package.json file.

Feels a little backwards that you specify dev on module install but prod on app install. Maybe --production being the default and --dev being an option makes more sense, but that would be one hell of a breaking change.

[orf]

isn't that what devDependencies is for?

[nathancahill]

"npm install" installs devDependencies too. You can do "npm install --production" to skip them.

[joshschreuder]

Anyone tried ied on Windows? I wanted to give it a go but it seems like there are a number of problems with permissioning / the symlink system [1].

That led me to trying pnpm [2] which, on the plus side can install my dependencies (and fast!), but on the other hand seems to do so in a way that my grunt tasks no longer work (complaining about missing sub-dependencies).

[1] https://github.com/alexanderGugel/ied/issues/35

[2] https://github.com/rstacruz/pnpm