[esprehn]

Dan isn't the first person to notice:

https://www.voitanos.io/blog/don-t-be-alarmed-by-vulnerabili...

We disable [1] audit entirely because it's not a good default behavior within a monorepo. It spams the hundreds of developers with the list of "vulnerabilities" on every install, but only a few folks should really be upgrading packages.

We then run audit in non-blocking CI and track the total number of issues and mostly focus on critical ones.

[1] https://docs.npmjs.com/cli/v7/using-npm/config#audit

[x0x0]

We also wanted to use npm audit in our CI process so, instead of humans being careful, we could assert any known CVE stops a staging or prod deploy. Very annoyingly, npm audit doesn't have ignore functionality, at least when I was last forced to use it. I had to hack something together with bash scripts.

For vulnerabilities that we determined weren't an issue ever (vuln in frontend framework we didn't use), or weren't high priority enough to P0 through, we needed some way to ignore either permanently or temporarily specific vulnerabilities.

Given the enormous dependency sets eg react create, you'd think the tools would be better at managing them.

[dane-pgp]

> Very annoyingly, npm audit doesn't have ignore functionality

I can't believe no one here has mentioned the fantastic tool "better-npm-audit" which can be included as an npm dependency[0] and lets you add specific vulnerabilities to an ignore list.

The ignore list is actually a JSON config file stored alongside package.json in the repo, so only one developer ever needs to see the npm audit warning and can mute it for everyone else (after getting their PR approved).

Even better, the config file lets you specify an expiry date for each entry in the ignore list, and provide a note, such as a link to the upstream issue being worked on, so that you can periodically be reminded to go back and check if a new version is available which can give more confidence that your code really isn't affected.

I think that developers might have to be instructed to use the "--no-audit" option to "npm install" if they don't want to see the (false positive) warnings that the default behaviour produces, and that's a bad habit to learn if not all projects they work on are using "better-npm-audit". I don't know if there is a way to make that option the default on a per-project basis.

[0] https://www.npmjs.com/package/better-npm-audit

[x0x0]

Thank you!

edit: we're going to move to this. My implementation is a is a file with cve to ignore | reason | jira ticket but I think we can slam all that in there. And our expiry is manually checked monthly, so this is a big improvement.

[underwater]

Wouldn't you want to only stop a deploy if the commit introduced the vulnerability (i.e. the deploy changed the dependency tree).

From my experience most audit flags happen because a new vulnerability is discovered, which means stopping a deploy doesn't actually do anything helpful.

[x0x0]

we do daily deploys, so we're mostly using this as a way to check cves across rails and react in a way that fails loudly and doesn't require anyone to step outside their standard workflow. Regardless of whether the new commits introduced the cve or not.

[prionassembly]

Having hundreds of developers work on the same code repository seems insanely complicated. What are the advantages? Where can I read more about monorepos?

[marcosdumay]

They do not use an usual git repository. Companies using monorepos have tools to restrict people's access to parts of it, and filter log noise.

Or, in other words, it's not that insanely complicated, because they have tools that make it look a lot like multiple repositories. And the large companies using multiple repositories have tools that make them look just like those monoreps. And HN has all those interesting threads full of people saying why one is better than the other.

[hinkley]

The feature I missed about Subversion was, if you could figure out how to treat your code like a proper tree, individual teams would check out one, maybe two directories, and only the leads and operational people ever had the whole thing.

I never entirely understood what it was in subversion's internals that prevented it from being used as a DCVS. We could figure out version numbers with branches and multiple repos. If not then, certainly now.

There's a space between subversion and Git that could be occupied with something that sheds the worst behaviors of each and makes something better.

[enneff]

Revisions in Subversion are identified by a monotonically increasing number. How can you distribute that?

[hinkley]

Versions are a tree structure. The implementation may have required the next revision number to be n + 1, but the data structure does not necessarily have to do that. If myself and someone else push commits, they have to be rebased and renumbered anyway, right?

[gervwyk]

We are just 2 devs at the moment and we've setup Lowdefy with a monorepo and yarn 2 and I can add that even for small projects a monorepo is just bliss especially with yarn 2 and lerna!

We come from separate repos on past versions of the code and the monorepo setup has sped dev up dramatically, even for a team of 2. Having the docs next to our code and autogen some of it from the source code, all is the same repo is just one advantage.

It's a small project but you can view the setup here: https://github.com/lowdefy/lowdefy

[SOLAR_FIELDS]

Google is probably the most (in?)famous example of monorepo. All of their code is in a single monorepo.

You can read a bit about the approach’s benefits and drawbacks from someone at Google here: https://medium.com/@Jakeherringbone/you-too-can-love-the-mon...

[x86_64Ubuntu]

Does monorepo mean that if I checkout Google Wave, I also checkout the GMail source too?

[esprehn]

Sort of, the google3 monorepo is so large that all checkouts are sparse checkouts. So you have a virtual checkout of everything but specify the subset you actually need.

Once you have the tooling to do this you can use the monorepo to store all kinds of interesting things, like built artifacts or the contents of the CDN, since it's basically just a giant hierarchical KV store with a global version.

[spullara]

Yes. And if you improve something used by both of them and have to refactor across the source tree you can do that in a single commit.

https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...

[suhas_rd]

yes

[esprehn]

There's lots of writing about monorepos these days, and they're quite common in public projects though usually at a smaller scale (ex. React is a monorepo, so is Babel and VSCode).

While complicated, in my experience the tooling to get the same productivity from hundreds of separate repos is at least as complicated and generally discourages folks from contributing to codebases outside their own.

Our monorepo encourages a collective velocity culture where everyone is pushing every project forward. If you upgrade React or Node, fix a security issue, or implement an optimization, the hundreds of apps are all improved at the same time. It's harder for one team to "leap forward" in terms of tech stack quickly, but at the same time it's far less likely we end up with hundreds of outdated or abandoned codebases since everyone is collectively improving the repo together.

For example I've rarely seen a single engineer upgrade a thousand separate projects across a thousand git repos with complex nested dependencies very successfully, but that happens every day in our monorepo.

[mulcahey]

Monorepos are pretty common nowadays, arguably popularized by Google

https://research.google/pubs/pub45424/

That said (1) “you are not Google” as the saying goes (2) Google built their VCS from scratch in house for scale.

However it’s not tops hard to do this with just git and some rules/patterns for organization.

[randomswede]

> Google built their VCS from scratch in house for scale

It's more complicated than that. Initially, there was Perforce. Then, Google grew client-side tooling on top of that ("g4", rather than "p4"). Then, someone grew client-side Git on top of that (for reasons, good or bad). And at some point, the Perforce backend simply stopped scaling, and a new backend was written (named "piper", but not the "piper" most non-Googlers think of). And once that was in place, assorted more "make it scale" tooling could be built on top of that (and, at that point, I stopped using the client-side Git, because CitC was simply That Good).

So, "from scratch" is a bit of a stretch.