|
|
|
|
|
by wwwigham
2403 days ago
|
|
Over at the TypeScript compiler we recently slowed our release cycle from 2 months to 3 months specifically because of an observation noted here: nobody used our beta (or RC) builds. On a 2 month schedule, we had 1-2 prelease builds during any given 6ish week period. The faster releases we had before were great for us - if something wasn't ready, we could just sit on it till the next release, since they were so often. However because they were so often, we struggled with collecting feedback on prereleases - we always, consistently, got most regressions reported only after the full release. We didn't really like this so a few things were tried. First we added an earlier prerelease cut and feature freeze (the beta) - this made the release smaller (as the earlier freezes meant more time focused on regressions or on the next release), and we still didn't get any feedback. Some of our users gave us the feedback that they'd test against our betas, but only if we had less of them... So we lengthened our release cycle to try that. We've not done many on the longer 3 month cycles yet, so I can't say if it's helped or not yet, but from the feature development side, I can definitely say that the longer releases definitely slow down how quickly things are built on top of one another. In parallel, we've also been improving our test infrastructure in ways similar to Crater. In the past few cycles we've added a bunch of tests which only exist to test compatibility - all of DefinitelyTyped is now tested on our builds, and a number of community projects that we've produced build scripts for are built as well. I would _love_ to be able to crawl GitHub and just build arbitrary stuff, but TS/JS build chains are almost never a simple `npm run build`, so the best we can get is, approximately, loading a repo into an editor and checking for a crash (which we do have a tool doing). The rust ecosystem's dogma around using `cargo build` and `cargo test` to handle _everything_ really does help make what they're doing possible. |
|
It is pretty standard in the Rust community for crates to use stable, beta, and nightly in their CI. I know testing is different than development, especially for using new features but I'm sure it does help.