[jesseryoung]

The author calls out a few reasons why DevOps fails for organizations all of which I agree with - however the one that I've never completely understood: Regulatory reasons for keeping Ops centralized.

I work in healthcare which I guess should fall under this rule - but in practice I haven't really seen that impeding DevOps. Teams that have the capabilities to build the full stack get handed a subscription to a cloud provider and they go off and do so. They still fill out and track change logs, audit changes and seek approvals - but after that's done, it's still the team who presses "the button".

Anybody in a regulated industry where you've hit hard walls that prevent you and your team from going full on DevOps? If so, what rules were quoted that stopped you.

[jgworks]

I am not in a regulated industry, but we have recently gone through the process of getting SOC2/ISO27001 certified.

This is what was cited for us.

ISO27001:2013 A.6.1.2: Segregation of Duties. Conflicting duties and areas of responsibility must be segregated in order to reduce the opportunities for unauthorized or unintentional modification or misuse of any of the organization's assets.

[randerson]

Surely that means that no one individual can push a change they created without involving someone else, but that it is still fine as long as any two people (even if they're on the same team) are involved? You could solve this by e.g. forcing GitHub to require a review.

[jesseryoung]

What exactly is a "Conflicting duty"? What's stopping a company from stating that developing, deploying and supporting software is a single duty?

[jiveturkey]

Nothing ... except compliance.

The idea comes from finance -- to require collusion to execute a fraud. It's not perfect, but it's something.

[jesseryoung]

Maybe I should rephrase that: Is it impossible for a company that defines Dev+Ops as a single responsibility to be compliant?

[jiveturkey]

Not impossible. even in a prescriptive framework like ISO 27001, adequate SOD is a judgement call between you and the auditor. Generally speaking, if a single dev can push a code change to prod, in a way that would escape audit or not require a second pair of eyes, that would not be compliant. So if a dev writing code, also manages the deploy environment, that may not pass muster.

But it's not that cut and dried. There are degrees of rigor.

[darkr]

No. Assuming a well configured continuous deployment type environment; you just need to have peer review on code before it can hit production, and you need to have controls in place over the who, what and when of elevated access to production being granted

[destroy-2A]

This all breaks down as soon as audit realise the Devops team is also admin of the ci/cd stack and therefore all controls put in place to make it harder for a single actor to do bad stuff can be bypassed via this all powerful system.

[Aeolun]

It seems like the description is vague enough that this is entirely up to whoever gives you your certification.

[devonkim]

There’s another reason that’s a bit older but there’s a line item in section 404 of SOX called “segregation of duties” which many bureaucrats interpreted to mean “developers must not have access to production” when that’s not what the regulatory requirement means. It essentially means checks and balances for accountability and auditability. If nobody can cowboy code their way into prod it’s fine. In fact, rogue ops engineers modifying code in production is an example of how separating ops and dev won’t really protect from insider threat vectors either. What really must happen is that there is a sure way to verify that code is approved by another stakeholder for deployment and tracked at traceability levels appropriate to who can fix it or should be able to view the info.

When people keep yammering on about devops as a principle of people and processes they’ve already lost because processes are meant to replace people, so really all that matters are the processes and the services that fit into the process SLA and OLA.

Note that in a big organization what really matters are your particular regulators and arguing with your regulators claiming to know it better than them is probably one of the fastest, reliable ways to get fired I can imagine that won’t result in a criminal lawsuit against you.

[zwischenzug]

Author here. That's interesting, as I've not worked with healthcare too much.

Others here have cited segregation of duties, which is definitely a factor, but the other one less mentioned in finance is the 'one throat to choke' principle: it's simpler from a management and regulatory perspective to have the responsibility for failures in one place rather than across many teams.

[jesseryoung]

Ah - that makes sense. This might be a bit easier in healthcare as I believe it's pretty common to have many different ops teams each responsible for different parts of the business.

I feel like most of the time "compliance" is blamed when really, it's your first point in that section (Absent an existential threat, the necessary organizational changes were more difficult to make) that is the real holdup.

[tibbon]

PCI doesn't _stop_ us from distributing these duties, but it sure does make it harder. Having change management processes in place puts in place all sorts of additional controls. Sharing code with this system and the main system creates either friction, or a lack of DRY code.

[robertlagrant]

Same here - made and make health apps, ISO13485. No requirement for a central ops team.

[jiveturkey]

SOD - Segregation of Duties.