[cthulhuology]

I this had happened at any of the companies I started, I would have fired several people in the aftermath. The problem is the policy is broken by design. Let's look at what is wrong here:

1.) priority is not just high, it is critical, communicating this is lost at each layer (executive, planning, execution, process control, quality control) 2.) leadership is lax, the chain of command doesn't designate a clear single responsible individual 3.) policy enforcement in this example actually increases the risk of an unsatisfactory outcome, by increasing the complexity of the solution vs. what is in production 4.) quality control is adversarial and ass backwards, code review is supposed to be a sanity check "does this code do what the developer thinks it does" aka. "can some other person understand it" 5.) test planning should not be the developer's responsibility, quite frankly if QA can't figure out if it is working or not you should fire your QA department. 6.) Ultimately, it is a total failure of policy and management as it requires the President of the company to micromanage the situation.

If you think any of this is fine, I'm sorry but your company is doomed to fail (unless it is already so big it is too big to fail).

[kevinstubbs]

You would fire them, instead of trying to build up their skills? I'm not even sure who you would fire in this situation, the IT director for not shepherding the ticket all the way through, or Shirley for closely following company policy? Either would be overreacting.

Slow/inflexible company policy has been built up, presumably under the IT Director's watch. The president didn't communicate that it was extremely urgent (and maybe it wasn't - the article doesn't say when the change actually needed to be made by to make a difference). Ideally, management can recognize that this change took a week because of their accumulated decisions, and lack of urgency. They should figure out exactly how their company policy is slowing them down instead of letting heads roll.

[user5994461]

Don't fire. Don't touch anything.

Just remember the chapter 12 of Peopleware.

Between two organizations, the productivity can vary by 10 times. It's not about you, it's a social characteristic of the organisation.

Just leave and find one whom speed suits you.

[solatic]

Specifically regarding your code review and QA comments, code quality is everyone's responsibility, not just management's or QA's. When development gets the idea that only QA is in charge of quality, development tends to write sloppier and sloppier code.

The real reason why this one-line change took so long is because the bureaucracy hasn't been automated away yet. Policy like not allowing magic numbers shouldn't be stuffed in a document, it should be enforced in a linting tool, and that tool ought to be runnable on the developer's workstation. If you have a QA department, QA should be spending its time writing automated tests, not maintaining and running manual tests. The developer ought to have been able to take 5 minutes to checkout the codebase, make the quick change, run the tests close to the change, and then submit to CI, which would run through the entire test suite and then continuously deploy to production. QA's approval should be built-in to the codebase and CI. Even in big companies, the process should not take longer than an afternoon.

[st3v3r]

"I this had happened at any of the companies I started, I would have fired several people in the aftermath."

And then no one would ever try to block a code review ever, and your code quality would go in the toilet. Then, something like this would happen again, except with a much less benign change, and your servers blow up (not literally).

"1.) priority is not just high, it is critical"

Priority is always critical for whatever little changes management wants.

[profmonocle]

> Priority is always critical for whatever little changes management wants.

Which is why in an actual emergency, management needs to actually communicate with the people involved in the development pipeline, not just hope the normal flow will magically work faster.

This used to happen constantly at my office:

Boss: I made a ticket to [change thing]. It's urgent, and please let [coworker I pass tickets to] know that it's urgent too.

...

Me: Coworker, I just sent you a ticket, [boss] says it's urgent.

Coworker: Everything is urgent. Shrugs and goes back to working from the top of his ticket list, all of which have the same high-priority flag as the one I just sent him.

[profmonocle]

> priority is not just high, it is critical, communicating this is lost at each layer

This was my exact thought. Why did a "fast tracked" ticket spend two days in a queue? Why did the requests to IT not also get set to urgent priority? (An IT request that blocks an urgent code change should itself be considered urgent.)

Most bizarrely to me, why the hell did they not defer updating the code to meet newer company policies? For the hardcoded variable thing, I would've said "ok, just change the value now, then create a separate, non-urgent ticket to bring that code up to our current standards."

It's sounds like their bureaucracy isn't equipped to deal with fast-tracked changes. The worrying part that management fed a critical issue into that bureaucracy and didn't prod it along until they noticed the delays. They may not fully appreciate how complex and full of roadblocks their process has become.

[edblarney]

1, yes. 2, not really 3, no 4, not really 5, yes, but does not affect this case.

If this company runs into this kind of problem often, then something can be done, but if this is a rare occurrence, it's not so bad.

Remember dev did not 'waste' 6 days of his time, rather, it took that time for the cogs to move together on it.

And even with 1 ... it's not super clear.

[ScottBurson]

We're actually not sure about (1), as 'kevinstubbs points out.

(2) It does appear that David, the IT Director, was responsible, though he was out of town during part of the process. (What? In 2016 "out of town" means "not reachable"?)

It's on (3) that I totally agree with 'cthulhuology and disagree with you. Changing the value of an integer constant in code is far less risky than what the code reviewer demanded. As others have argued upthread, the original one-line change should have been put through to production as it was, and the additional changes should have followed later via the established process... if indeed they were truly necessary at all. (As someone else pointed out, a variable whose value changes once in 10 years probably doesn't really need to be a configuration parameter, though making it into one probably counts as a slight improvement.)

For (4), okay, here I don't entirely agree with 'cthulhuology. There is a place for enforcing standards; code review isn't just about comprehensibility.

[bzbarsky]

> What? In 2016 "out of town" means "not reachable"?

The text explicitly says David's signature was needed. Unless they have an e-signature setup, or he can locate a fax machine, it doesn't matter how "reachable" he is.

(That's assuming that "out of town" doesn't mean "on vacation", in which case "not reachable" is in fact quite reasonable.)

[anon1209]

> (As someone else pointed out, a variable whose value changes once in 10 years probably doesn't really need to be a configuration parameter, though making it into one probably counts as a slight improvement.)

I'd argue that adding a variable that changes once in 10 years is not an improvement. Config files should consist of things that actually need to be configured. Adding crap you'll never touch makes it harder to find the variables that are actually important, and also makes it harder to read and understand all available options.

Introducing configuration variables for the hell of it is analogous to writing every method to be as generic as possible; useful but not worth the risks and expenditure. Just like how we often draw the line and say "this method doesn't need to be any more generic", config variables similarly need an approach more nuanced than "move everything to the config file".

[edblarney]

"Changing the value of an integer constant in code is far less risky than what the code reviewer demanded."

This depends on context.

An integer variable can kill humans if it's the wrong variable.

Was it related to safety?

Who knows?

I'm assuming the reviewer knew what they were doing ...

Maybe some common sense all around would have helped here.

[frumiousirc]

It's the president's fault for letting this problem rise to 10% and risk layoffs before acting. Like most leaders, his psychopathy is visible by the wake it causes.