[observationist]

At one of the top tier 1 ISPs in the world, there was a bastion host that allowed 2 teams of network engineers unfettered access to everything; once your permissions allowed you access to the bastion, you had everything. 50 some people with trivial credentialed access to network infrastructure that the world ran on; fatfinger a bgp config and you could take down countries. Swathes of cities were regular casualities of config mistakes, and if you locked yourself out without setting a reload in 5, it'd take an hour to get someone deployed.

That experience shattered my idea that the world was being operated by competent engineers and technicians, governed by sane policies, under the watchful care of good, knowledgable people.

The world is held together by beliefs and expectations and bubblegum and duct tape, and a few thousand people madly scrambling to keep it all running.

[nextaccountic]

> That experience shattered my idea that the world was being operated by competent engineers and technicians, governed by sane policies, under the watchful care of good, knowledgable people.

Reminds me the amount of debt that exists only as an entry in an excel spreadsheets somewhere. No database with high availability and regular backups and audit logs and access control and all of that, just a spreadsheet.

[icedchai]

Sounds like the 90’s early ISP experience scaled up. No firewalls, everything on public IPs, text files with global credentials in clear text…

[jasongill]

I have been transported back to the days of `conf t`, `enable password hunter2`, `show run`, `copy run start`

[icedchai]

And the days of hubs, not switches, telnet and rlogin, not SSH. A hacked host could potentially compromise your whole LAN.

[esseph]

A lot of that is still there in various ways.

[htrp]

> The world is held together by beliefs and expectations and bubblegum and duct tape, and a few thousand people madly scrambling to keep it all running.

Sounds like the AWS experience