[datadrivenangel]

The bit about Knight Capital implies that the software engineers were bad, which is notably untrue.

"A bad [software engineer] can easily destroy that much value even faster (A developer at Knight Capital destroyed $440 million in 45 minutes with a deployment error and some bad configuration logic, instantly bankrupting the firm by reusing a flag variable). "

[charlieflowers]

Both the article's examples there are bogus -- yet in both cases the underlying points are true.

Google generates a lot of revenue per employee not because the employees are good (though many of them are of course), but because they own the front door to the web. And the Knight Capital story has many nuances left out by that summary.

In both cases the author needed a hard hitting but terse example. But as I said, both the claims are true, so in the voice of the courtroom judge, "I'll allow it."

[hinkley]

There were decidedly shitty engineering decisions behind that dumpster fire.

The biggest being that the only safe way to recycle feature flag names is to put ample time separation between the last use of the previous meaning for the flag and the first application of the new use. They did not. If they had, they would have noticed that one server was not getting redeployed properly in the time gap between the two uses.

They also did not do a full rollback. They rolled back the code but not the toggles, which ignited the fire.

These are rookie mistakes. If you want to argue they are journeyman mistakes, I won’t fight you too much, but they absolutely demonstrate a lack of mastery of the problem domain. And when millions of dollars change hands per minute you’d better not be Faking it Til You Make It.

[datadrivenangel]

The powerpeg feature flag had been deprecated for 9 years? In aggregate, yes, the engineering led to the disaster, but the specific engineers had been making everyone piles of money for a long time. The mistakes were fatal, but in a tiny amount of time on a system that old it's honestly surprising it didn't happen sooner.

https://specbranch.com/posts/knight-capital/

[hinkley]

Deprecated is not removed. I’m talking about airgapping EOL and resurrection of a feature flag code.

The fact that they hordes old code for NINE YEARS evaporates most of the remaining sympathy I had for them. Witaf.