I say this as someone who only uses AWS for DR backups: hosting infra is one of the most competitive industries out there. Major cloud costs are exorbitant but that should be interpreted as proof of a great product. I shudder at paying 9c/gb egress but apparently many others do not.
> hosting infra is one of the most competitive industries out there.
Here is an experiment - tell your employer you will be hosting on (insert no-name provider here), to same a literal million dollars, and see if you can get security team to sign off on it.
Here is another experiment - reach out to he security team, and tell them AWS costs are too high, ask them which providers they will be ready and willing to sign off. My guess is it will not be a big list beyond AWS/Azure/GCP.
So the market is not competitive at all, most of us cannot switch providers even if the alternative would be 100x better.
PS: I am not saying security team are assholes, I am pointing out a major barrier to competition.
> PS: I am not saying security team are assholes, I am pointing out a major barrier to competition.
> Here is an experiment - tell your employer you will be hosting on (insert no-name provider here), to same a literal million dollars, and see if you can get security team to sign off on it.
???
So what does this have to do with the security team at all? There is no "barrier" in that sense.
In the past we've had more non-cloud engineers than cloud. Using your experiment, if you told your IT team you wanted to move to the cloud (back then) to save a million dollars - do you think they'd sign off on it? No.
Who signed off on it? The bosses that believed in the "hype".
Who's in control and who has power? If the bosses want it to happen it will even if it doesn't make sense. They have the ability to fire the security team if they said no. Just like how ethical AI teams get fired...
The barrier is those in power still believe in the "hype" and don't know otherwise.
I met a CTO of a startup sometime ago that moved their entire operations from GCP to AWS because they were "more familiar with it". That's all.
> I met a CTO of a startup sometime ago that moved their entire operations from GCP to AWS because they were "more familiar with it". That's all.
Without knowing which startup you are referring to its hard to make a judgement as to the quality of the decision but you should not discount the role tooling familiarity has when developing software.
> but you should not discount the role tooling familiarity has when developing software.
No 1 was familiar with the cloud when it 1st came out.
As to this scenario, clearly the whole company was running GCP so everyone minus the new CTO would be familiar with GCP vs something else.
Point exactly being that regardless of the security team or the developers - this familiarity that you mention or any other trait only applies to a select few in management.
> Here is an experiment - tell your employer you will be hosting on (insert no-name provider here), to same a literal million dollars, and see if you can get security team to sign off on it.
SOCS/PCI/etc is going to take maybe $100-200k. If you can save a million dollars you should do it. Hire an expert if you have to. Serious.
I think it probably won't save you a million dollars, because I think all of the cloud vendors are priced with just enough profit to make sure of it, but if you know something I'd like to know about it.
Speaking as someone who went through this process at a large financial firm, you're off by at least an order of magnitude. You need a SOC1 audit of each product you plan to use, which is likely quite a few if you want to take full advantage. The big players should eventually be able to offer that for free once they've been through the process but, at least relatively recently, it was only true today if your cloud budget was tend of millions. That aside, you'll still need an audit of your usage of the cloud (i.e. how you deploy to it and handle movement of data back and forth). That'll always be on your dime.
No. At the end of the day, the customer pays, because I charge more for bullshit. They need an SOC to use my cloud product it cost me 150k USD to get an audit from a big-four for a single site in 2016. Maybe it’s a little more today, but it’s not an order-of-magnitude.
I’m assuming you already adhere to the relevant standards. Obviously if you’re cutting corners getting up to snuff is going to cost a lot more than a hundy.
A Big 4 can't conduct a proper SOC audit without access to the cloud providers internal controls/processes. That's the problematic/expensive part since it requires a bunch of time from the cloud provider, which they will also likely want to bill for.
As someone currently dealing with SOC in preparation for the company I work for going public, I will also confirm it is a giant bean-counting pain in the butt.
> I think all of the cloud vendors are priced with just enough profit to make sure of it
Profit margins on cloud computing are insanely high (at least, relative to my expectations). They basically have no interest in anything with less than a 15% margin, even at the massive scale they operate at. Certain products have triple-digit margins. Even if they are the minority, I don't think we can give them a pass with claims of "just enough profit".
The reality is that there’s myriad providers that simply do not provide the assurances that AWS/Azure/GCP do. Sure, there’s a bit of “use these, because we know them, and they work”, but there’s also a bit of “the typical developer is not at all across the security requirements, especially taking into account contractural obligations and regulated industries”.
tl;dr even after getting a big public sector contract, a UK based cloud provider was killed off after scaling to meet demand which was then withdrawn. Attaining - and keeping - scale is extremely difficult. And that was just IaaS provision.
If you're scaling your whole business for one customer based on one contract then you better be sure you can either scale down again, or the contract has safeguards in it to stop a rugpull.
The magic of cloud is how quickly you can scale things up. If you're a new fast growing business it will give you a competitive edge.
If you drank to cloud kool-aid from the beginning you kind of get used to huge costs for simple services so it's easier for your brain to justify paying 10k a month for a simple web app deployed in kubernetes, using cosmos db and any other number of services.
I've seen many companies that started on the cloud and their core architecture is so interleaved with the cloud that it would a huge investment to reduce that dependency and switch.
> The magic of cloud is how quickly you can scale things up
I see this spouted a lot, but my recent (last 6 months?) experience with AWS is that unless I pay up front to reserve a tonne of high end instances that I don’t necessarily need today, but might need tomorrow, I’m regularly running into capacity issues where I cannot spin up new instances of the metal that I want, and AWS support confirms they just don’t have the capacity unless we pay to reserve it up front.
At that point, it’s no different to running my own DC, where I already have 3 months of runway on my server pipeline anyway.
I wonder what the cause of that is? I think I've heard this sentiment elsewhere recently but I don't recall it in the past. So what's caused the capacity constraint?
The risk to Amazon of enabling you to scale up low end instances on demand is relatively low.
The risk of doing the same with high end instances is a different story.
Low risk & high margin products make for a highly profitable business. High risk and lower margin products less so.
Their target audience is people that can be equally well served by digital ocean and their ilk, but are happy to pay the Amazon premium.
It’s a good business model for Amazon, and a terrible value proposition for the customer who may not know better, and thinks they’re paying to de-risk their potential future growth requirements. The cost of migrating out then becomes prohibitive (both in technical hours and egress fees), so you like it or lump it, but either way, you likely just wait and/or pay Amazon more.
Same. I use Hetzner and DigitalOcean for my own stuff, and shudder at AWS costs. At the same time both my current and previous employers use AWS and it's fine for their user because revenue is (very) high relative to the cloud resource usage, and that is a situation where cloud usage is fine. But so many - B2C in particular - businesses who uses cloud have tight enough margins that they're just setting themselves up for being disrupted by a competitor with tighter cost control.
Do people even understand what "free market" means? A free market is one with low or no barriers to entry for suppliers and perfect information available to consumers such that they can always make the optimal choice. How can anyone, especially on this site, think the cloud market is anywhere close to being free?
I think people are confused thinking free market means no regulation. No regulation leads to monopoly which is the furthest from a free market you can possibly get. People who seriously advocate for such things are ignorant fools. There are countless examples of how markets fail and that's why every major economy in the world has a government.
It depends on what you mean by free market, it's often used interchangeably with with laissez-faire capitalism but an important concept in a free market is that the barrier to entry for new competitors should be as low as realistically possible, so regulation that prevents vendor lock-in can be pro free market.
It makes sense with this being an American site, and most Americans think that Socialism is essentially the same thing as Communism (but then why define them as separate things?).
Community, Cooperative, Employee owned are all forms of socialism. None of them prevent competition.
> no market incentive to provide a decent service, nor to compensate workers fairly.
If you owned part of the company you worked for, you would be more likely to provide good service as that would then have repeat business and would directly financially impact you. The performance of the business is the only way you would get paid and as there are no shareholders the company would compensate rather than dividend.
Your comment makes absolutely no sense, socialism doesn't prevent free markets.
Socialism is about social not state ownership of the means of production. This does not prevent the state being involved either, and even then they can be a joint-stock corporation's.
As I said, "social" ownership of the means of production can occur in a capitalist country with free markets.
A socialist regime, like the USSR, North Korea, or Mao's China would have state ownership. The founding fathers of modern socialist thought (e.g. Marx) would define socialism in terms of state ownership of the means of production.
But feel free to muddy the waters if you prefer to avoid concrete definitions.