[thesmart]

I think the convenience of GitHub and NPM are good features, and it's easy to blame ease-of-use as a culprit. I have often pondered about how remarkably poor these tools are at representing repo quality. I think there are some basic questions to ask before accepting any unknown source in as a module. What is the core problem and requirements the source addresses? Does it verify the solution and how? What is the reputation of the core contributors? Is it used by any serious institutions? (Aside: download count and star count are a measure of herd mentality) What are the open issues? What are the closed "won't fix" issues? How often are issues regressed? What is a safe version? How is the versioning managed and how should I pin it? How many dependencies do I assume from this library? How severable are the dependencies?

It would be useful if package management tools facilitated the process of understanding the liability imposed by a package, but the opposite is encouraged. Authors too often put up shiny marketing materials and make bold statements about the utility and vision of their software. It would be refreshing if authors were as open about the flaws, trade-offs, alternatives, etc. but few are.

Perhaps some kind of rating or feedback system that is qualitative in nature would help mitigate the salesmanship? I'd love to know who has been burned by a project and anecdotes about how packages are used by others. Ratings around issue resolutions would also be helpful, how often have we all had major bugs dismissively closed by maintainers?

TL;DR there are ways to develop a comprehensive assessment of a repo, but our tools are lacking and need much improvement in this regard.

[extrapickles]

I would like this idea extended to versioning, where you get a simple release number and a risk number. Even for projects following simver, they all have a differing idea of how big a change is and simver doesn't cover bugfixes that make breaking API changes.

If a new version simply incremented the release number, and estimated the risk of issues from the previous version, it would be easier to tell if you wanted to update. Also after a project has a few releases under its belt, you can normalize the risk to other projects risk level (eg: project x always underestimates).

For open source projects, it enables tooling to look at the functions touched and the functions used by your code, and add risk to the update.

An example:

You are 3 versions behind, updates are versions 15/42 (minor bugfix), 16/500 (new minor feature) and 17/32000(major api change). Risk of updating is 42+500+32000. If you peg the amount of risk for a automatic update to 1000, then you would only get the first two.

The same thing in simver: 1.0.1 1.1.0 2.0.0

While its somewhat easy to gate simver, it doesnt lend itself to automated risk assesment as it would be much harder for a tool to tell the difference between a bug fix and new feature (too many bug trackers out there).