I find it strange that senior developers would laugh at the idea of using quality control gates in a CI/CD pipeline. I know a lot of people don't bother with these things, but most seniors I've worked with would at least aspire to (or say they aspire to) using such techniques.
But I guess that's the difference between a senior who knows lots of syntax, and a senior that that knows how to deliver quality.
Senior here, 20 years experience, started out with C++ projects where the only quality gate was the build server failing and currently working on a project where commits fail because of lint rules including things like imports being alphabetized.
IME the best approach is a "balanced" one. Let devs push their branches from local to the repo without passing tests or lint rules. (e.g. git push --no-verify). Try and keep your tests fast so devs want to run them locally. (Carrot, not stick). But prevent a merge request from being merged to your trunk branch until everything is green.
Personally I find "linters" horrible. They're like someone with OCD leaning over your shoulder while you code. I think VS Code actually runs a LOT slower when the linter is integrated, which can do more harm than good. How crazy is it that TypeScript can do all its advanced type checking faster than the linter?
Yes, completely agree. Also alphabetizing includes in C++ is just wrong. The more basic file should be below the less basic file. This is because the less basic file may need to include the more basic file itself and the compiler no longer complains about the missing include if it appears above it.
OCD & often wrong pretty much summarizes linters for me.
The only caveat I'd add is that it depends on the project and the team how many gates you want to add. A couple of super-senior devs building a small blog aren't going to need many gates. A team with 50 people, half of them juniors, working on a financial app already in production - they're going to need a lot of gates.
As with all these things, ask the team if they're adding value. If they are great, and is there anything else that would help? If they're not, and especially if they're unnecessarily slowing things down, then don't do them. Don't use a code style checker because somebody in a meetup said you should.
In my experience, any project with excessive automation and such rules quickly turns into a game of "find how to pass the checks" and you spend the majority of the time fighting the bureaucracy rather than doing useful work.
If your devs don't care about the quality of code they deliver, they'll find a way to deliver whatever crap they've built - even if it takes longer to find a workaround than just write some good code.
CI/CD should not be used to force developers to do things they don't want to do, but to automate some easily repeatable tasks they would still do anyway.
Yep, forcing MR's to be rebased to the last master/develop commit helps a lot here. IIRC this is a feature you can enable in Gitlab. Much less chance your MR will break master/develop if your work is rebased on the latest commit.
Bors doesn't force you to rebase. Forcing the rebase is super annoying in projects with more than a few developers (I've worked on one that did).
What bors does is essentially the same thing, but automated: after all the tests have run on your MR branch, and people have approved your changes, bors does a speculative merge into master, and runs the tests on that merge commit.
If the tests pass, bors declares that very same commit to be the new master.
The effect is the same as the Gitlab option you mention, but the busy work is done by the bot.
I assume these tools are configurable, so devs could agree what's the balance. We usually start with defaults (assuming somebody put effort into choosing the defaults) and customize as we go.
Pushing without verify and fix everything at the end is worse than not fixing at all imho. After all fixes you basically have a lightly refactored code that has been less tested.
I've worked for many companies that had fully automated CI/CD pipelines. If you have a good team which cares about the project, then you don't really need them IMO.
The reality of most big corporations is that nobody actually cares about the company. Executives can't trust their engineers to run the tests manually and deliver something that works.
If you have a good hiring and promotion process and a fair compensation structure (which none of the big corporations have), then you can trust your employees and so you don't need to micromanage them with automated CI/CD pipelines. In addition, if people actually care, you get much higher quality, more succinct code.
No amount of automated checking can lead to improved code quality. It only ensures that the absolute minimum operational requirements are met at any given time.
> If you have a good team which cares about the project, then you don't really need them IMO.
I strongly disagree. It's not about caring or not, it's about making mistakes.
A human can easily forget to run some test, or generally one step out of many in a complicated procedure, especially if they're new to the project. A reviewer can miss a mis-formed identifier. The more is that automated, the fewer manual steps need to be remembered, and the more safety the project gets. Automated tools like CI/CD should not be considered supervisors to humans, but helpers and safety nets.
That being said, there are certainly cases where teams don't set such tools up because they come with an implementation cost. And depending on project complexity and their use case in general, it may be more economical to do things manually.
Not using automatic tools means using a person to do a computer's job.
Programmers should work on interesting tasks: refine and revise the tool rules, add even more automatic checks, refactor and correct genuinely flagged code.
Refusing automatic check tools means lowering happiness, dignity and productivity, wasting attention and time with bad activities and priorities: following boring and error-prone procedures, finding ways to elude rules and take shortcuts, low-value and tedious detailed code reviews (e.g. discussing indentation instead of incorrect edge cases).
You're right that if your team is filled with people that either don't know or don't care about quality, then no amount of automation is going to help. If corporate thinks an automated CI/CD managed by a separate operations team is going to bring quality, then, yeah that's always going to fail. But that doesn't mean all and any automated CI/CD pipeline is not going to help.
I've both worked with and implemented CI/CD pipelines, as in a Jenkins script that builds a number of build artefacts and then promotes them to run in a particular environment. As part of the build process, it runs automated tests and static analysis on things like type checking, code style.
What it doesn't do is tell us which tests we forgot to write so there's no 100% cast iron guarantee that everything is correct, and I think that's what you're referring to as "improved code quality". But by having that automated step, our human QAs can spend their time on manual exploratory testing (which humans are very good at) rather than manual regression testing (which is extremely slow and inefficient). And our devs can focus code reviews on questions like "is this code clear to me?" rather than "did you meet the in-house code style?" or "have you forgotten about potential nulls?". And we can bring in contractors (who might not care so much) or juniors with a fast onboarding time, since a lot of mistakes will get picked up automatically.
If you're working in a sufficiently large organisation, one where independent auditors look at development processes, then you must have a CI/CD pipeline - either a slow, manual one or a fast, automated one. I know which one I'd prefer.
> If you have a good team which cares about the project, then you don't really need them IMO.
Of course as senior I do want them if I care about project. It makes it less likely for me to forget. It eliminates one of the "procedural chores" that I have to remember to type on command line.
I care about the project and therefore I do not assume that I am unfailing perfection of never making mistake or never skipping a step.
Yes, there's only so much attention a human can pay to little details. Unburden the brain, so you can worry about subtle interactions in your code, instead of catching every last trailing space.
"No amount of automated checking can lead to improved code quality." - Very untrue. The more automated checks (customized to your needs), the less distraction for devs and more focus on things that matter, e.g. quality.
Wherever I worked, these kind of checks did always more harm than good. You force a bunch of tools on your developers which they hate, only to have your whole codebase littered with "disable linter X" comments. In a good review process the developers can negotiate the conventions by themselves, which leads to acceptance among the team.
Well if the team doesn't like the linter tool they should be able to change it. That's a political problem and nothing to do with linters.
My experience is that linters (that people like) saves you from having to constantly discuss formatting, minor problems at the PR stage which is otherwise a huge time sink.
I can think of a few ways of how this could backfire / be gamed or abused, but all in all, seeing how everything else is just as susceptible to abuse and gaming, it still sounds awesome to me.
But everything was upside down with that team, not just payment. They had a completely different philosophy when it came to software development.
For instance we were encouraged to just close tasks. It didn't matter what we did as long as the build and code reviewer were happy.
Part of the philosophy was that if developers can abuse and break the project, the project is not good enough and must be toughened. Then, developers were encouraged to find other loopholes.
This methodology worked wonders for small projects, libraries. It didn't go well for big projects, obviously.
This sounds identical to the agency I was reading about.They pay devs for closed tasks. I.e.: a class needs to written that does x. The reward is $100.A senior dev will do it half an hour, while a junior will spend a few hours,which bases work-reward process self balancing based on experience.The owner of the shop had some very detailed blog posts on this,which attracted tons of discussions.Shame I can't recall the guy's name..
It was fun work back in the day of Teamed.io. But nowadays, with Zerocracy, it's turned a bit too strange even for my taste, after having worked about 2 years with Teamed.
If the project isn't a huge bureaucratic nightmare, you can fix those loopholes as they come up. You might even give a small bounty for people who report loopholes. (But make that bounty mostly bragging rights, not money.)
I have to say, I did not encountered such situation. Teams I was in used linter and equivalent tools and the "disable linter" directive was basically never used.
The rules to be used were negotiated among developers. In one team, the use of linter cut down on the uneven random unforcement of the unofficial rules that was going on rampant (and subsequent frustration of involved people).
That's just a matter of methodology and/or discipline.
Depends on the team: if the team is democratic, then they should reach an agreement beforehand and respect it. If it's a dictatorship, then just punish them for not respecting the rules.
If the devs just agree to break the rules and disable them, they should be punished somehow. "Punished" is a big word maybe, but that's the idea :)
Nice article - one nitpick is that I absolutely do /not/ make the checks mandatory as part of the build process. Rather there are some "opinionated" checks that are run when code is pushed for review (which can be skipped), and then strong checks run as the first step in the CI/CD pipelines. This means that the tools stay out of the way when developing and running locally, but still enforce standards where required. This leads to fewer developers disabling or @ignore-ing rules to test things locally and then forgetting to remove them.
This article also reminds me of a talk I saw from Neal Ford about architectural fitness functions: the idea being that if there is an architectural pattern that should be followed then the best place to put it is in an automated step as part of your CI/CD. I thought it was interesting to take the concept normally limited to linters and apply it to a more abstract principle
I think it depends on the check. Some checks are better enforced strictly in an automated fashion. Style rules are a good example: they make no performance difference, lead to bikeshedding, and are pretty much arbitrary. Other checks should be made automatically but not strictly enforced. EG you might want to require all C casts to have a comment describing why it's safe. That can be a good linter warning, and linters tend to have codes that can be added in comments to suppress the warning. So the linter warning can make code review easier, but shouldn't necessarily fail the build.
Great, so now I have 100,000 duplicate bug reports to sort through because I forgot a null check and the logger automatically created an issue each time that error occurred (with different timestamps, causing duplicate detection to fail).
This sounds nice on the surface, good intent. But it smacks of inexperience and is the same mechanism that ends up pushing the kinds of dumb conventions that insist on golang's Trailing Comma. What is the expected outcome of your rule, and how do you expect additional, incremental bureaucracy to drive that outcome? Any time you push rules, you must expect that people will default to simply obeying the rule without questioning why the rule exists in the first place. This is just a natural cost-minimizing function of human beings. As a former console operator of a decades-old mainframe, let me warn you that this is how informal "operator manuals" end up spanning many three-ring binders filled with nonsensical, sometimes contradictory trivia, and how you end up with people going through the motions like some kind of ritual whose origins are lost in the mists of time.
If you're going to do something like this, I suggest that these kinds of "code quality gateways" aren't as absolute as OP recommends. Be smart about it: You could probably measure how "sloppy" a developer is (relative to some mean of their own output and not compared to the team or really anyone else). Simply bringing it to a developer's attention (programmatically) will do far more for your project's code quality than trying to dumbly enforce some list of code conventions about whitespace or whatever. Also, this is a kind of tool that should absolutely be scoped strictly on the level of the individual developer, with no possibility of having externally-visible reports generated from it, and absolutely not exposed to management. You don't want to give management too many knobs to turn, especially knobs like this.
>Writing about this now, I got an idea: we should have some sort of plugin for Log4J or slf4J: a plugin that would automatically open tickets on Github or other trackers, when the .error(...) method is called
Great idea in theory. It will not fly in practice. If you tell to any enterprise company that your (or their) software is automatically doing this behind the scenes, the project will be shot down.
I mean, getting crash metrics is one thing, but opening automated issues from enterprise software is a completely different manner.
I like the approach Visual Studio App Center (and probably others) takes of aggregating crash reports so you can easily figure out which issues are impacting the most users and prioritize.
Thanks for the comment. I am a sucker for spelling indeed. I usually spend about 1h after publishing each post, to fix typos (I never notice them until it's published, no matter how hard I try).
But I guess that's the difference between a senior who knows lots of syntax, and a senior that that knows how to deliver quality.