[sgarland]

> Developers need to have ownership and accountability for their infrastructure without having to become absolute cloud experts.

This will never happen. You can’t own something you don’t understand.

Ops and Dev are different roles for a reason, and the only reason we’ve shifted away from that is to accelerate profits; yes, you can spend your way to growth, and yes, you can run massively complex systems on hardware you have never seen, nor understand. That doesn’t make it a good idea.

[dwill-mdcloud]

I am unsure what you are getting at. Our platform is all about Ops and Dev being separate jobs, but operating at a greater scale. Ops produce modules that can be consumed by Developers. Developers control the scale of their infra while Ops encodes security and compliance in the module. It seems like this would be the ideal model in this instance. If it is not, what does the ideal look like in your opinion?

[sgarland]

Just as choosing the correct algorithm, the best library, or the optimal design pattern for a given task is complex and important, so is choosing, configuring, and managing infra.

The hyperscalers have convinced people that you don’t need to know how to run a database, you can just use RDS et al. You don’t need to know how to manage K8s, you can just use EKS. This is dangerously untrue, because those tools are good enough that most people can get them going and they’ll work reasonably well, right up until they don’t (especially RDBMS). Then you hit edge cases that require you to have a solid understanding of their architecture, as well as Linux administration – for example, understanding how Postgres’ bgwriter operates, and how it is affected by disk IOPS and latency, not to mention various kernel-level tunings. None of this matters in the slightest with small DBs, say, < 100 GiB. It may not even matter at larger scales depending on your query patterns.

The various DB offerings (I’m going heavily on the DB example because that’s my current job) like Neon and Planetscale mostly have the right idea, IMO – stop assuming devs know or want to do ops work. They want an endpoint, and they want things to be performant, so have automatic schema and index reviews. Critically, in this model, devs are not responsible for nor accountable for the infra’s performance (more or less; discussions on schema design and its impact on DB performance aside). In other words, this has separated ops and dev.

I say they’ve mostly got it right because they do still allow you to make bad decisions, like chucking everything into a JSONB column, using UUIDv4 as a PK, etc. Realistically a service would fail if they tried refusing to allow that flexibility, so I get it.

For an in-house solution, though, this can be the case. The old school, rigid mentality of having to ask cranky graybeards to add a column had an extremely powerful benefit: bad decisions were much less likely to occur. Yes, it’s slower, and that’s a good thing. Everywhere I’ve been, bad decisions made in the name of velocity have come calling, and it’s a nightmare to fix them.

In summary, I like the idea of Massdriver quite a bit, actually; I just don’t think it’s a good idea nor accurate to say that it allows devs to be responsible for their own infra, because they largely don’t want that, nor are they capable. Not for lack of intelligence, but lack of experience. Let specialists be specialists. You want a DB? Ask the DB team, and don’t get mad when they tell you your proposed schema is hot garbage. You want more compute? Ask the infra team, and don’t be surprised when they ask you if you’ve profiled your code to determine if you actually need more.

[dwill-mdcloud]

That's a fair assessment of the problem. I appreciate the thought you put into it. I think compiling all of those experts will be difficult for companies until they are fairly large. We hope to push out requiring that expertise for as long as possible. But in an ideal world, I don't disagree with you.