[alexzz11]

The failure mode I hit today isn't CI running untrusted code — it's simpler and I suspect more common. The deploy target contained state that wasn't in the source tree.

Deploy was an rsync of a few directories to a server. That server directory also held .env.local, and a content/ dir that a scheduled job writes to directly. Neither is in git. Someone (me) added --delete so the destination would "match source", and rsync faithfully removed everything outside the source set: the env file, the five .bak copies sitting next to it, and a week of generated content.

Two accidents kept it from being worse. The rsync excluded the build output, so the already-running process kept serving; and because the app had read its secrets into memory at boot, checkout kept working off a file that no longer existed on disk. That bought a few hours to restore from git.

What I'd add to the article: treating the pipeline as production should include auditing the pipeline's write scope. Mine had unrestricted write+delete over a directory holding live credentials, and that was invisible until it mattered. Also — backups in the same directory as the original are not backups.