[joncrane]

I've been working almost exclusively in the AWS space for about 10 years now. Clients anywhere from tiny little three-person consultancies to Fortune 100. Commercial, govcloud, dozens of clients.

Never once have I ever found a use case for making public EBS snapshots.

Who on Earth is thinking that it is a good idea to take an EBS snapshot and make it public?

Note, several of those engagements did involve multiple accounts, and the need to share / copy AMIs and/or snapshots between accounts. But never making them public.

[dboreham]

Laziness in attempting to share data with someone in another org?

"Nope, can't access it" ...

"Nope, still can't access it"...

"My manager is harassing me to get access now"...

"Look, just make it public then change it back after I get it copied"...

[privateSFacct]

This 100%. I don't do much AWS stuff - but this issue exists outside of AWS as well (inside a corp).

It's why I hate all the password rotation, double VPN stuff - people work around it too much.

All the old staff start having the tech person track their passwords or write them near the computer, all the young staff use whatever new .io domain does X super easily (but less securely) or stick things on thumb driver (no keypad) or use their personal google drive etc.

[heyoni]

goes home completely forgetting to change it back...

[hinkley]

The later in the day, the bigger the rationalizations.

At some point you are too stupid to be allowed to use a keyboard. And yet we have an entire culture around staying late to get stuff done that probably could have waited until tomorrow.

[joncrane]

Yeah, that's pretty believable. Guess some places have a laxer culture and no automated scans checking for things like that.

[somepig]

most places

[Bombthecat]

Too real...

[lowdose]

The guy that produces that last line definitely wears a suit.

[atoav]

As much as I dislike the typical suit wearer, the industry as a whole would be much better if IT-people wouldn’t blame everything on them and just tell them you won’t do it. If you are a welder and you boss wants you to weld a gas tank that hasn’t been emptied you tell him that he has no idea what the thing he asks for means, explain why and wait till the gas tank is emptied and everything is checked.

I worked as a Camera guy in film and I am known to be very fast – yet I had directors who wanted things even faster. Bit there is a natural limit to how fast you can get something done without having worthless garbage as a result.

You can take certain risks, skip certain advisable steps, focus on the most essential thing etc, but below that there waits literally nothing.

In my experience taking a step back, breathing in, out, and then proceed to doing it properly is in most cases faster than following your boss into panic and ditching common sense.

It is your responsibility as a professional to say “No” or “Stop” in certain circumstances. And if they really want you to do it, write down the possible consequences of what they force you to do and make them sign that they take the full responsibility.

There is so much talk about IT security with people frowning upon silly behaviour, yet any other craft would bend over backwards before you could force them into unsafe behaviour. If engineers would build bridges like we in IT operate, we would have many collapses a day.

[tialaramex]

(Not advice to parent, just generally)

If you want to be more professional about this stuff, build up a Fuck Off Fund. Women in particular have written about fuck off funds in the context of making sure you don't have to nod along when HR says the VP who stuffed his hand down your top was "just fooling around" but - most people need that financial security any time they have to confront the boss. Save to be able to look the big boss in the face and tell them "Fuck Off". "Fuck Off" isn't the response you need when they tell you they want the database authentication disabled "just until Monday, Tuesday at the latest" that's when you want "No". But you need to know you _can_ tell them to "Fuck Off" so you actually say "No". Otherwise you may find yourself agreeing anyway.

[yourapostasy]

For the youngsters out there, don't elaborate when you say "No". That is, don't mention your FOF. That weakens your position.

It is sufficient to look the manager/executive threatening you square in the eye, and state your position with deliberateness. Keep it professional, no raised voices, and be willing to walk away without hesitation if the other side gets abusive.

The really bad ones are those who tell you that if you step through that door don't bother coming back or similar, so be absolutely ready to commit. If you do the main event behind closed doors one-on-one and get that threat as you walk out, sometimes they'll come back with sugar suggesting a do-over. Generally Admiral Ackbar is right in this context, but it's your call.

The negotiation leverage that comes from the FOF is most powerfully communicated non-verbally and in a face-to-face setting, also through body language. The difference is very noticeable between those who have an FOF and those who don't, if you have enough experience. It can be faked, but it takes exceptional practice to fake. The tell starts with how fast and confident the "No" comes back.

This is the nuclear option of course. Exhaust all other avenues of reasonable negotiation first. Like an email with witnesses you pick for deliberately violating departmental policy, for example.

[thegeomaster]

>If engineers would build bridges like we in IT operate, we would have many collapses a day.

Maybe a bit offtopic at this point, but whay you said reminded me of this: https://www.stilldrinking.org/programming-sucks

[jackcodes]

I really enjoyed this, thanks. Do you have any other similar ones, or tangentially related bookmarks you could share?

[ygra]

It reminded me a bit of James Mickens’ writings. Not quite as weird and insane as him, but close.

[nurettin]

All of these anecdotes have something in common. That the person who is asked to do something is informed about proper procedures. That is uncommon in IT, and especially in companies where proper procedure is too costly and not worth pursuing.

[atoav]

It is still a communications problem. They want you to do the minimum. If I as a camera person do the real minimum the director will be pleased on set but they will get a heart attack in the editing room.

It easier to give in, take short cuts and produce garbage than explaining them under extreme stress that everything faster will result in nothing.

In my eyes it is comparable and I worked in both industries. The difference is, that as a DOP in film your name will be associated with the mess on the screen, while in IT the link is not that clear. This makes the difference.

And unless your director has a monitor (which they sometimes don’t) they cannot tell at all what you are doing. They wouldn’t even realize if you didn’t even switch the camera on.

I know that film or engineering is more

[fixitfixit]

Oh no. I work with very trendy Silicon Valley folks and it’s always someone in shorts and a t-shirt. Which is to say: it’s definitely a culture issue, but it doesn’t have to be a PHB doing it. I know a lot of really smart people who are under a lot of pressure, and combined with a lack of AWS knowledge, “just make it public” is surprisingly common.

[missingcolours]

Nah. "Move fast and break things" usually has an implicit "and bypass all the security, we'll fix it later"

[deadmutex]

This seems like a bias

[lowdose]

It is and Nassim Taleb has frame it as the expert aka the empty suit problem.

[deadmutex]

To be clear, I didn't say the bias was with the person wearing the suit.

[t34543]

Aren’t public EBS snapshots the underlying mechanism for public AMIs? I’ve ran into complex permissions in a golden image deploy model where the same AMI is used across multiple accounts.

There needs to be controls like S3 where you can explicitly block public data.

AWS IAM kind of sucks.

[spydum]

I’m convinced the last few years of ramped up concern about AWS “blast radius” is an admission by AWS professionals that NOBODY gets IAM right.

[d2mw]

> Who on Earth is thinking that it is a good idea to take an EBS snapshot and make it public?

Non-marketplace AMIs are built on public EBS snapshots, but that's something they should still fix. Marketplace AMIs already handle private snapshots

[based2]

to synchronize backups to the onprem datacentre. https://angus.readthedocs.io/en/2014/amazon/using-ebs-snapsh...

[Exuma]

Is it a default setting that it's public or do you have to go out of the way to do that?

[fromthestart]

>Who on Earth is thinking that it is a good idea to take an EBS snapshot and make it public?

Maybe they're trying to reproduce functionality of docker? It would actually be extremely useful for research involving modeling/AI because you could trivially reproduce the results by bundling the exact code and data.

Edit: actually maybe I'm confusing EBS snapshots with AMIs...