[madelyn]

I mean, possibly? What's considered the gold standard in package management these days?

I use yarn for managing javascript dependencies and do a lot of work with Cargo too. The community seems to love both these tools outside of slow compile and install times.

[throwaway894345]

Cargo is my ideal, but really anything that doesn't make me manage virtualenvs or take 30 minutes to resolve dependencies. Note that "managing my own virtualenvs" is tricky because you have to make sure everyone has all of the same versions of the same dependencies in their virtualenv across your entire team (including production). I'm sure there are workflows that allow for this (probably with some tradeoffs), but we haven't figured it out. For a while we used Docker, but performance degraded exponentially as our test base grew (Docker for Mac filesystem problems, probably). Eventually we settled on pantsbuild.org which has a lot of problems, is super buggy, no one can figure out its plugin architecture, etc but as long as you stay on the happy path it generally works okay which puts it in one of the ballparks between any other Python dependency management scheme I've tried and Go/Rust/etc package management.

[stuhood]

Great experience report: thank you! Wanted to point out that the Pants project has been focusing on widening that happy path recently (...by narrowing its focus to Python-only in the short term), and is ramping up to ship a 2.0.

This page covers some of the differences between v1 and v2 of the engine, and particularly its impact on Python: https://pants.readme.io/docs/pants-v1-vs-v2 ... We're using Rust and haven't bootstrapped yet, so we also appreciate Cargo and think that there is a lot to learn there.

We'd love feedback (via any of these channels: https://pants.readme.io/docs/community) on how to make it even better. Thanks!

[throwaway894345]

That’s great to hear. Is there any page that documents the architecture of pants? I understand build systems of various kinds quite well, but I can’t tease out the design philosophy behind pants, especially how the different target types / plugins end and the “core” begins.

I’m going to dig into that 2.0 link though!

[stuhood]

There isn't, but that's a good idea. At a very high level, the v2 engine is "Skyframe but using async Rust". All filesystem access, snapshotting, UI, and networking is in Rust, and we use the Bazel remote execution API to allow for remote execution. The v2 plugin API is side-effect free Python 3 async functions (called `@rules`) with a statically checked, monomorphized, dependency-injected graph connecting plugins (ie, we know before executing whether the combination of plugins forms a valid graph). Unlike plugins in Bazel, `@rules` are monadic, so they can inspect the outputs of other `@rules` in order to decide what to do.

This file contains a few good examples of `@rule`s that collectively partition python targets to generate `setup.py` files for them automatically: https://github.com/pantsbuild/pants/blob/5e4f123a1dbc47313fe...

[pas]

yarn. fast and correct. every package.json is a virtualenv, but you still have a global cache and it just symlinks. (sort of like the global wheels cache for pip, but even more deduplication.)

cargo is great because it manages the build flow, it's extensible (clippy) - the global cache thing is a bit harder, because of rust package features (and other knobs like RUSTFLAGS), and it's not done by default, but it's as easy as setting RUST_TARGET_DIR as far as I know.