[yunong]

Even more important, npm doesn't reliably reproduce builds. Different runs of the same shrinkwrapped build could either fail or succeed -- which is odd given that builds should be deterministic. Additionally, shrinkwrap is also broken in the same way.

This is the first time I've heard of ied, a little competition could go a long way!

[krisdol]

Could you expand on that? What causes it to be nondeterministic? I haven't had that experience but it's the second time I've read someone reference this behavior

[dandelany]

Here are details from the docs: https://docs.npmjs.com/how-npm-works/npm3-nondet

[kemitchell]

> You can reliably get the same dependency tree by removing your node_modules directory and running npm install whenever you make a change to your package.json.

https://docs.npmjs.com/how-npm-works/npm3-nondet

[krisdol]

That explains why I haven't experienced it -- our projects' npm scripts to build packages wipe out that folder before building.

[yunong]

I have a project that's shrinkwrapped. I check out a new copy of the project and run `npm install`. My co-worker does the exact same thing, his fails, yet mine successfully builds. An install of a shrink-wrapped should be deterinistic, i.e. either succeed or fail, but not both.

[Wintamute]

From that doc page it sounds like that should be deterministic? Even without shrinkwrapping, a fresh install from package.json with empty node_modules should be deterministic.

> The npm install command, when used exclusively to install packages from a package.json, will always produce the same tree. This is because install order from a package.json is always alphabetical. Same install order means that you will get the same tree.

> You can reliably get the same dependency tree by removing your node_modules directory and running npm install whenever you make a change to your package.json.

Maybe you're experiencing a bug, rather than some in-grained non-determinism in npm?

[madeofpalk]

My understanding is that even when you fix your dependencies to exact versions, your dependencies probably haven't so without shrink-wrap you'll never know _exactly_ what gets installed.

[kemitchell]

What's the error on the build that fails?

[juandazapata]

That's frustrating, I got bitten by that a couple of times. I'm wondering when will they add a LOCKFILE like every other build system out there :'(

[tlrobinson]

That's what npm-shrinkwrap.json is supposed to be. It's given me a lot of trouble though.

[mbell]

I agree, but I'm curious what other build systems have a lock file?

Ruby does through bundler, who else does this right? I can't think of any.

[stormbeta]

You get a similar effect for free with any language whose community defaults to non-floating transitive dependencies, e.g most JVM languages.

[technomancy]

Indeed; a lockfile is just a workaround for the problem of "just give me whatever" dependency declarations. Solve the problem at the root rather than piling hacks on it.

[dasil003]

I'm not 100% sure from your comment, but are you suggesting that by default you should vet every version exactly and freeze it at the point you defined it?

As someone writing a lot of ruby, "give me whatever" is analogous to "give me the latest unless I specify otherwise", which I consider to be a very good default. It keeps me up to date with security issues, and incompatibilities between libraries that the respective maintainers resolve amongst themselves with a minimum of manual work.

[technomancy]

> are you suggesting that by default you should vet every version exactly and freeze it at the point you defined it?

Yes, this is the way that Guix and Leiningen and rebar3 and a bunch of other things work, and it is wonderful.

Pulling in new code without you asking is a fine idea for something like apt-get where you have a huge team doing QA on the entire system working together before it even hits your repositories, but for most package managers, the dev team is the one doing the testing, and upgrades should be done only with great care.

It does mean you have to watch for security updates, but this is true of all package managers.

[caoilte]

We use Ruby extensively in testing and infrastructure and have lost months of cumulative developer time to this attitude. It works for a single user constantly making changes to a small piece of code and prepared to work out the solution to dependency change problems as soon as they come up. However, it doesn't scale to a team of people working on very large code bases.

One example of this is having a repeatable developer setup guide. If the dependencies might have changed by the time a new joiner starts your setup guide could very easily end up useless or misleading and that's without anything at all in your own codebase having changed.

Shrinkwrap fixes this, but always gets added after things went wrong several times already. It should be the default.

[mbell]

I've spent enough time in my life fighting with version mis-matches of slf4j. Hard defining your dependency versions just pushes the problem upstream.

[sync]

Mix, Erlang's package manager (used by Phoenix / Elixir)

Cargo, Rust's package manager

Meteor

[SEMW]

> Mix, Erlang's package manager (used by Phoenix / Elixir)

Slight nitpick: Mix is part of Elixir, I've never seen a (non-elixir) erlang project that use it. Don't think erlang has an equivalent officially blessed tool, but the most popular one is rebar / rebar3.

(While I'm nitpicking: mix and rebar are more build & dependency managers; mix uses hex for package management. I believe rebar3 can also use hex. In ruby terms, mix ~ bundler (among other things); hex ~ rubygems).

[strmpnk]

These come to mind: rebar3 (Erlang), mix (Elixir), composer (PHP), elm-package (via elm-stuff/exact-dependencies.json).

[josegonzalez]

I try and push devs to do

    pip freeze > requirements.txt

Which has a similar effect.

[jonafato]

This is a good start, but it has issues. You probably don't want all of your locally installed packages in a requirements.txt file. Instead, they should be curated. I've been using pip-compile [0] for a while now (see the author's blog post [1] for a detailed explanation) and have become a big fan of it. With this model, you should enumerate only what your application uses directly and let pip-compile convert this list to a full version-locked requirements.txt. (Shameless plug: I also wrote a bit about why this is the best currently available option for specifying Python dependencies [2].)

[0] https://github.com/nvie/pip-tools#example-usage-for-pip-comp...

[1] http://nvie.com/posts/better-package-management/

[2] https://www.jonafato.com/2015/12/15/Rethinking-requirements-...

[jsjohnst]

If one uses virtualenv as a Pythonista should, then your keep point doesn't carry much weight.

That said, this looks cool and I'll check it out.

[jonafato]

This should be used with virtualenv, not as an alternative to it. I have a bunch of libraries and tools in almost all of my virtual environments that are not application dependencies and have no business being installed in my production environment (e.g. bpython, tox, flake8, and neovim). This approach also handles things like a dependency being dropped gracefully (if some library no longer needs a dependency, it magically gets removed from your locked requirements.txt next time you compile requirements.in). Python's package management tools are making great strides (see pip's recent peep-style hash checking support), and pip-compile is a big win in that category in my opinion.

[rodgerd]

And it's so awesome when you discover devs have used lockfiles to leave you stuck on known-insecure gems instead of updating their fucking code to support the newer patched version.

[mikekchar]

Which is still better than having your server crash due to a bug in a dependency of a dependency that got updated without your knowledge when you deployed something trivial. It has happened to be more than once.

[juandazapata]

Cabal (Haskell) too.

[geerlingguy]

Composer (PHP) too.

[theknarf]

PHP have a package manager called Composer which have a lock file.

[lintiwen]

same issue here. this non-deterministic behavior is really fxxked up. there's one time that our building process suddenly begin to fail, spend a few hours on the issue, and it turned out to be one of the babel-core patch release is broken.

some of the very fundamental designs of npm is seriously wrong.