[PacifyFish]

What kinds of things do you need to check for that aren’t automated? Maybe I’m spoiled by our CI pipeline.

I’d love to take a look at your list

[vortico]

Here's a somewhat redacted version. `m` is my alias for `make`.

    - Finalize source
     - Legal check `LICENSE-dist.txt`
     - Update `CHANGELOG.md`
     - Bump version in `Makefile` and `Core.json`
     - Commit "Bump version".
    - Build (three OS's can be done in parallel)
     - `m clean`
     - `git pull`
     - Make sure you have the latest `Fundamental.zip` package in source root.
     - `m dist`
     - Manually test installer and fragile features (audio drivers, patch loading) for ~10 minutes.
     - `m notarize` (on Mac)
     - `m upload`
     - `git tag vX.Y.Z`
     - `git push --tags`
    - Release
     - Update version title and URLs in `Rack.pug`
      - `m upload`
      - At this point, normal users have access to new version.
     - Update server version in `config.coffee`
      - `m restart`
      - At this point, normal users will swarm to download new version. Keep an eye on server bandwidth.
    - Publicize
     - Twitter https://twitter.com/home
     - Facebook https://www.facebook.com/vcvrack/
      - Share on group
     - Forum https://community.vcvrack.com/c/announcements

[jl-gitlab]

Interesting list, thanks for sharing. I'm the PM for CI/CD at GitLab, and I don't mean to hijack the topic, but wanted to give a heads up that we are looking at building a feature for these kinds of procedures that aren't quite pipelines, based on Jupyter Runbooks. Would love to hear your feedback: https://gitlab.com/gitlab-org/gitlab/issues/9427

[vortico]

Currently the most difficult and most time consuming step in the release checklist is finding space on my desk for three laptops so I can use them all simultaneously. Releasing only takes 1-3 hours, and I unfortunately wouldn't want to spend hours learning and setting up automated methods to perhaps shave 10 minutes off the process.

Manual releasing allows me to iteratively improve the process and spot possible issues, whereas any automation would reduce build reliability.

[judge2020]

> whereas any automation would reduce build reliability.

I guess this depends on what your release process is; for example, most npm libraries benefit from CI that runs the build script and runs `npm version $TAG && npm publish`. There's not much that could be improved here, especially if the build script is just an alias for running `webpack`.

[scarejunba]

There's a method I read about on lobste.rs for this. You write down the list as you did and then turn pieces into "Do X by running Y" and over time more things get automated.

That's a hefty deploy protocol, though, and automating cross-platform builds may be more trouble than it's worth if you're releasing rarely.

It's pretty neat you follow a deploy checklist.

[rgoulter]

Related to the process of "write down what you did", I'm reminded of this article about "literate sysadmin/devops" which used Emacs' org-mode as a way of combining prose, commands and command output. http://howardism.org/Technical/Emacs/literate-devops.html

[maccard]

I read you should write it as an interactive script e.g.:

    input("Build X")
    input("Build Y")
    input("Zip X & Y")
    input("Upload release")

THen run the script and press enter as you do the steps. It means if you're interrupted as your proceed, you can resume exacly where you were before.

If you implement it as functions:

    def buildX():
        input("Build X")
    def buildY():
        input("Build Y")
    def ZipUp():
        input("Zip X & Y")
    def Upload():
        input("Upload release")
    if __name__ == '__main__':
        buildX()
        buildY()
        ZipUp()
        Upload()

Then you can automate parts piecemeal.

[enriquto]

this list is a very simple algorithm. Why don't you tun it into a script deploy.sh ?

[tbrownaw]

Some of the steps look like they'd be hard to do that for. Like for example the one about making sure the license is correct, and the one about testing parts that apparently don't have good automated tests / interact with the real world.

[enriquto]

> Some of the steps look like they'd be hard to do that for.

Then those steps must be removed. I thought the mantra "deploy is ONE step" was a more or less universally acknowledged truth.

[creatornator]

I mean unless a computer can check that all dependencies have compatible licenses, that is unrealistic. It's a necessary step for publicly released software to avoid legal issues

[cyphar]

Linux distributions (though I can only speak to openSUSE's process) have automated scripts (which I believe are written by our lawyers) which check whether the license of a package matches the license of the files inside the package. It's how most package legal review gets done (and if the script can't figure it out, it gets escalated to our actual lawyers too review. You cannot submit a package to any one of the distributions we ship without the legal review being approved.

So it is clearly possible to do -- and there are all sorts of tools which figure out what SPDX license entries apply for every dependency (or vendored dependency) of a given project.

[udkyo]

I took the parent posts to mean that step involved checking the project license, not its dependencies.

For the latter though, Fossology, Scancode etc. can help.

[gdfasfklshg4]

Because it involves inherently manual processes.

[vortico]

No it's not.

[enriquto]

But it almost is, already! Half of the lines are literally shell commands. And except maybe for the legal check, the rest of the steps can also be written as shell one-liners. For example, to update the version on a README file you can "sed -i" with an appropriate replacement, and so on.

[vortico]

Can you explain how your ./deploy.sh script forks onto three laptops (Windows/Mac/Linux) and "plays around" with the installers/build a bit to make sure it works? I obviously can't release builds that don't even run because I didn't take a few minutes to test them.

[cobbzilla]

Sounds like you are mixing release & QA, makes a lot of sense in a small shop.

In a larger team you’d have a small agent on each of your N (likely >> 3) machines, CI pushes to the agent for build/release/automated tests.

Then if those pass on a given system, it fires off a message to start manual QA validation for performance and other “intangibles”.

[enriquto]

Why don't you use virtual machines?

[happytoexplain]

How have you come to that conclusion?

[anon9001]

I can do every step on your list with any CI platform. I do something extremely similar to build apps for ios (macos), android (linux), and web (linux). I think it's so important that it's one of the first things I do on any project.

I wanted to give you an example, but it all seems so easy, I'm not sure where you're getting stuck.

[gdfasfklshg4]

Can you really do a legal review from a CI platform?

[wwright]

If it can be automated, then obviously, yeah.

If not fully automated, many CI platforms allow you to pause the pipeline by requiring manual confirmation from an authorized user. I know Jenkins and CircleCI support this off the top of my head. You could have the CI perform any relevant searches or diffs, then display that to the user to get manual confirmation. It’s still not “perfect”, but it does allow you to reliably get someone to look at the data and say “yes” or “no.”

[anon9001]

Yeah, that's one approach. Another might be to commit some kind of `./legal/sign-off-v1.0.0` so you can see who did it, and so travis can know if it should deploy the build or not

[gourlaysama]

The Scala release checklist [1] is a good example of a release checklist with both automated and manual steps.

[1]: https://github.com/scala/scala-dev/issues/645