As in, no human should run a package manager in prod? (But salt/ansible/etc. running it is fine) Same idea as "if you're SSHing to prod, something is wrong" (where provisioning tools make all changes, logs are all aggregated and delivered in their own tool, and even debugging is built into the app or logging system).
More as, you should not modify images running in production. By human or machine.
Rather build new images and roll over the fleet. If you need to debug, remove from production (quarantine) and work on it there.
Don't run master / agent setups for ansible / salt anymore. You can still use them for creating images, which are later turned into running VMs. Think about it like containers. Do you update the contents of your running containers, or log into your containers to make changes?
But golden images on Linux are, well, messy. It's very annoying to make a clean VM template without some post-provisioning like cloud-init. And for most shops if you're running cloud-init you could do that post-provisioning with Ansible or Salt. And since your images are built with Ansible/Salt in the first place you might as well just build each VM fresh and use the vendor's ISO. One less thing to maintain and update.
Plus when you're in a pinch, which never happens of course, you can make changes without having to roll your VMs.
I feel like Atomic distributions are basically the happy medium between the two worlds.
It's a trade-off. The point is the matrix security lapse turned worse because they ran this master / agent setup. You can still use ansible (or similar), just do it localhost during the build process.
Yea it's easier to not do these things, because good security posture takes work to set up. Once you're on the immutable train, you'll find it's not actually harder day to day. You learn to deal with issues in the pinch another way.
On the point of building VMs fresh each time vs building golden images, you'll find you boot time reduced, your roll over more reliable and autoscaling more responsive. Why build the same thing dozens or hundreds of times? What happens if a remote package is updated in the middle of your upgrade? Does this sound messier to you?
So it's a less audited application than *SSH that the author is recommending over SSH because it doesn't require user authentication but runs in a daemon with root privs?
Not quite.
In this case Ansible is enabled by a user logging into the system; The user should not be allowed to login to the system in the first place. Ideally you want your configuration system indepenent of user logins.
Ansible has its place, my argument is that it doesn't belong here.
You should not be running package managers on production servers. Or any of the other things salt, ansible, chef, puppet can do.