Hacker News new | ask | show | jobs
by wwwigham 2403 days ago
Over at the TypeScript compiler we recently slowed our release cycle from 2 months to 3 months specifically because of an observation noted here: nobody used our beta (or RC) builds. On a 2 month schedule, we had 1-2 prelease builds during any given 6ish week period. The faster releases we had before were great for us - if something wasn't ready, we could just sit on it till the next release, since they were so often. However because they were so often, we struggled with collecting feedback on prereleases - we always, consistently, got most regressions reported only after the full release. We didn't really like this so a few things were tried. First we added an earlier prerelease cut and feature freeze (the beta) - this made the release smaller (as the earlier freezes meant more time focused on regressions or on the next release), and we still didn't get any feedback. Some of our users gave us the feedback that they'd test against our betas, but only if we had less of them... So we lengthened our release cycle to try that. We've not done many on the longer 3 month cycles yet, so I can't say if it's helped or not yet, but from the feature development side, I can definitely say that the longer releases definitely slow down how quickly things are built on top of one another.

In parallel, we've also been improving our test infrastructure in ways similar to Crater. In the past few cycles we've added a bunch of tests which only exist to test compatibility - all of DefinitelyTyped is now tested on our builds, and a number of community projects that we've produced build scripts for are built as well. I would _love_ to be able to crawl GitHub and just build arbitrary stuff, but TS/JS build chains are almost never a simple `npm run build`, so the best we can get is, approximately, loading a repo into an editor and checking for a crash (which we do have a tool doing). The rust ecosystem's dogma around using `cargo build` and `cargo test` to handle _everything_ really does help make what they're doing possible.

2 comments

> Over at the TypeScript compiler we recently slowed our release cycle from 2 months to 3 months specifically because of an observation noted here: nobody used our beta (or RC) builds.

It is pretty standard in the Rust community for crates to use stable, beta, and nightly in their CI. I know testing is different than development, especially for using new features but I'm sure it does help.

The TypeScript is usually install via npm, but like many package managers it doesn't allow multiple user-specific installed versions of a package.

(What package manager allows for your multiple rustc versions? Or is it a bespoke solution?)

Fortunately the (IMO superior) yarn package manager does.

    "devDependencies": {
        "typescript": "~3.6.0",
        "typescript-beta": "npm:typescript@3.7.0-beta"
    }
Instead of using node_modules/.bin/tsc, use node_modules/typescript-beta/bin/tsc
> (What package manager allows for your multiple rustc versions? Or is it a bespoke solution?)

rustup, the recommended / website default way of installing rustc, supports installing multiple channels. The command it installs on your path as "rustc" (and "cargo" etc.) is actually a wrapper; you can use e.g. "rustc +nighly foo.rs", change the default for your user, and even change the default for a directory.

That said, CI is typicality done in ad-hoc VMs or containers (e.g., a Travis matrix), so having different versions in automated test runs would be straightforward even if rustup didn't support this.

Slowing down the cycle is good for developers who like to keep their TS version up to date as a matter of routine. It means fewer updates to do per year.