Hacker News new | ask | show | jobs
by scarejunba 2349 days ago
Sys admins are dead. I have GKE now via a reliable Terraform module. None of my production instances can be logged onto.
3 comments

But if you run your containers on GKE you are a sysadmin. The system might not be a traditional UNIXy one, but there are still elements you need to take care of that aren't just application development:

Do your containers have security vulnerabilities? How do you keep track of them?

How do they communicate? What's your blast radius if something is compromised?

Who has access to your container management plane?

Have you correctly sized your system?

Is it resilient to failures?

What about your data? Is it backed up? Have you tested the backups? Do you have volumes that can fill up? I/O limits?

All those are sysadmin things that will never go away.

I understand what you mean, but your GKE instances probably can be logged onto: `gcloud compute ssh --zone=<node-zone> <node-name>`

Both COS and Ubuntu nodes are integrated with IAM, so this transparently provisions a user on the node, and copies a temporary SSH key. If your GCP user has the appropriate IAM permissions, they will also be able to use `sudo` to execute commands as root.

Are you sure about that?
Only time will tell if I am in fact right. I'm counting on being more right than the guys who have dedicated staff who routinely shell into their servers.

I suppose we'll see if companies with sysadmins have more breaches than the guys who run their own ops using container orchestration etc. I think I'd go even odds $1k that over the next five years, most large scale data breaches will be at organizations where sys admins run the majority of ops.

There's a whole new category of errors you can make (making your bucket open, etc.) with cloud providers but the tooling has better defaults.

> Only time will tell if I am in fact right.

I guess you prove a point here: you trust that the deployment of your GKE module allows you to be safe, so your investment vs risk trade seems to satisfy you. But you, yourself, cannot even predict how insecure you are at the moment due to the complexity of the software solutions you are using.

> I suppose we'll see if companies with sysadmins have more breaches than the guys who run their own ops using container orchestration etc.

Oh, I do agree, at least with the current state of sysadmins out there.

But the problem is that you loose control of the integrity of your system once you reach a point where software complexity becomes your security entry point.

If you are sure the containers you will be using are secure, have sane defaults, are up to date, etc, then fine, good job! I just don't trust that most people will be able to reassure me that. And please keep in mind that in your case, the target is not your container platform, the target would be the containers in that platform, and the services they run.

No one can really predict their security accurately.

Say you maintain a bare-metal server in a data center that your company controls.

How much do you know about its physical security? The protocols for admitting new staff? Do you rely on your company’s physical security team and HR? Are any of those functions contracted externally, even partially?

How much do you know about the network security? Do you rely on a networking team? Is any of their work contracted externally? What about the link to the outside world?

How much do you know about the sourcing of the physical hardware?

How much of the source have you audited? How about firmware source?

GKE obviously introduces new factors and vectors, but it also simplifies many of these and adds elements of herd immunity. And it’s also the same as the rest: your system was built by many people, it will be used by many people, and it will be maintained by many people. You can spend all of your time verifying every link, or you can help them do something in the world.

How well do you vet your IT staff? I was once the recipient of an inside job: http://boston.conman.org/2004/09/19.1
You're absolutely right about not being able to predict the security. But, to be honest, I doubt people genuinely do that in a scientific manner.

And yes, I do expect the failure point to be my containers, not the container orch platform. The container orch platform allows me to use containers, though, and the fact that containers aren't append-only the way that long-running machines run by sysadmins are gives me a head start out of the gate.