[eropple]

I gotta ask, because now you're firmly in my backyard--what's the biggest cloud spend you've ever been in charge of? I ask because "Cloud costs" keeps coming up as this bugbear reason to use k8s and it isn't a real concern for the 99th percentile of applications. An application that's expensive when running directly against a cloud provider's APIs will remain expensive when running in k8s, if not moreso because of k8s's steadfast refusal to pay attention to the bin packing problem. The galaxy-brain thought on HN is that cloud providers are so much more expensive than OVH or Hetzner or whatever--it's literally meaningless. People cost a lot. Even an inefficient use of AWS doesn't cost very much. By the time you're at the point where your cloud spend exceeds one FTE, you should probably have forty and the wins it gives you should be self-evident or you've screwed up somewhere else (and that "somewhere else" is probably your business plan).

I've moved nontrivial systems from AWS to GCP and in the reverse direction. It's a job done in Terraform/Pulumi and while a competently written application or set of services needs some work to do the move it's work you are likely to do once at most. (Emphasis on at most. The overwhelming, overwhelming majority of companies are way better off going multi-region in a single cloud provider than going multi-provider. Multi-cloud is for the rich and the silly.) The underlying cloud provider doesn't matter very much when you can pay somebody like me to come in for a month or two and help you make your application an actual citizen of the platform you want to use and leverage its efficiencies properly. The "good long term bet" is abstract interfaces in your code--the hype-driven cycle of the new-and-shiny means there's a nontrivial risk that k8s is no longer sexy enough to blog about by the time that "oh, we now need to move to a new provider!" even matters to you.

(I am contractually obligated not to step in the microservices pothole. It's a good way to waste development time and not ship, though.)

[rumanator]

> I ask because "Cloud costs" keeps coming up as this bugbear reason to use k8s and it isn't a real concern for the 99th percentile of applications.

AWS VMs tend to cost between 3 to 4 times more than equivalent VMs offered by smaller no frills service providers such as Hetzner and Scaleway.

You may argue that you don't mind paying a hefty premium for a service that has plenty of competing offers, or that some high-level service provided by AWS is nice to have, but it's hard to argue in favour of needlessly spending 3 to 4 times as much to provide the same service, or get somewhere around 30% of the cloud computing resources for the same price tag.

[eropple]

This post is exactly the sort of thing I see a lot. It belies inexperience with systems when they go bad. And systems basically always go bad. (I made a very good living off of that!)

You know what you don't get with a Hetzner--which is fine for what it is, this is impugning their customers and not them as a provider? You don't get the for-free metrics and alerting of an AWS. You don't get their sizing. You don't get their effectively-infinite inventory (EDIT: I was just reminded of a former employer who managed to make AWS tap out of a particular instance type in a region--how do you think Hetzner's gonna fare if you hit that kind of scale? Are you sure you built an effectively multi-region, multi-master system on your magic k8s cluster?). You don't software-driven infrastructure at every layer of your stack--sure, you get kubectl, now scale your cluster without waiting for hours for a human to rack a machine. Also without crossing your fingers every time.

(Scaleway is better about APIs and responsiveness, to their credit. But we're not out of the woods yet you are now the barrier to reliability and you're going to have to invent or duct-tape half of a cloud provider on your own to get something done. I too enjoy not doing things that help ship products, but not when I need money...)

You know what you do get, though? You get risk. You're worse at data integrity and backup than AWS is and Hetzner is worse at DR options than AWS; these are not exactly controversial statements, so I trust that you'll just go with it. But wait, there's more--you also get inflexibility. You get the risk of k8s itself--which is, let's be real, kind of a tire fire if you aren't Google, I've never seen a k8s shop of nontrivial size where something wasn't constantly alerting or broken. And you also get inefficiency. You get inefficiency of spend as you buy more hardware than you need and then pay the marginal cost of managing and alerting all of it, because you need overhead space--which is usually deadweight space--for when your hardware fails. (Because your hardware will fail, and you need a way to fail over.) And you get inefficiency of people; you get to waste the time of an expensive resource (or apply an incompetent one, which might be your thing but it sure isn't mine) reinventing the wheel over and over again. Sure, you can run a RabbitMQ instead of a SQS. I hope you know how to deal with its nonsense (I barely do, and I've run it at scale) and I hope you have an ironclad backup plan for it. And even when you do, you get the people-time inefficiency of building and maintaining and spending time and attention on things you get as part of your "hardware" spend with a full-featured cloud provider.

From all indications from over a hundred clients at sizes from five employees to fifty thousand, you probably are no exception to any of the above. And maybe I'm wrong about that, maybe you're the one exception who does. But I will always bet the other way, even when it's me and I know that I'm no slouch at this, and that's why I use AWS or GCP. (It doesn't hurt that, ultimately, it ends up being cheaper, both because I need fewer infrastructure/devops people to manage it and because I build systems that don't require one to run enormous servers--when you don't box yourself into the corner of "I need to run this big fat daemon all day long" you can spend remarkably little on AWS or GCP in the first place!

Maybe doing things the hard, slow, scary, and risky way is your thing, and you're willing to pay in time, effort, and risk rather than in money. But, one, it really isn't much money if you're running something successful, and two, it is downright disingenuous to imply that the spend is like-for-like.

[streetcat1]

Thanks again.

So more expensive is on a relative basis and taking into account the egress traffic from the cloud. My workload involve training machine learning models and serving them. Training on the cloud is 20X more expensive, and you cannot use commodity GPU (banned by Nivdia).

Moreover, My platform offer automl, which basically trade data scientist time for compute time. However, since I need to train 100's or 1000's of models, this can become very expensive, very fast.

Since I am not sure what my customers load will be like, I want to give them the option to move between clouds or on prem.

For the long term prospects of kubernetes. For me it is clear the kubernetes have found PMF and it is now at the first 1/4 or 1/3 of the S curve. IBM is all in, VMWARE is all in, Azure is all in, Gcp is all in.

Also, what are the alternatives ? Do you agree that containers are better than jar files or manual deployment options?. Do you agree that having a CI/CD pipelines with fully automatic unit/func tests is better than throwing code over the wall to some QA department?

So if containers are better packaging/deployment architecture, they need to be managed/monitored etc.

For micro services. In my case, the data since part is written in python, while the control plane (kubernetes operators) is written in go. So micro services are actually a very neat solution to a polygot product.

[eropple]

I used to work at IBM and I have first-hand experience with exactly how wonderful (that smell is not the dog, that is sarcasm) their Kubernetes implementation is; without mincing words, I would consider IBM's enthusiastic adoption of k8s to be a warning sign rather than a positive indicator. VMware, ditto, they're a trailing-edge company flailing for modern relevance. Azure is a cloud provider so persistently awful that they and IBM are the only ones I'd refuse a client on because it's not worth the frustration; not sure I'd bet on their thinking, either. And GCP--sure, the people who made k8s like k8s, that stands to reason.

Containers are fine deployment tools, sure. And Fargate is a better management and monitoring framework than whatever you'll roll together, while also not making you pay overhead for hot-spare and failover inventory. (Their prices used to be really wacky; if you dig through HN you will find a post from the day Fargate was announced where I looked at the numbers and had some Questions. I do not anymore.) I get that you have GPU stuff to deal with, and maybe that doesn't work for you--but EC2 instances probably do, can be sized better, and can be dynamically scaled without breaking your back. Things you don't pay for are cheaper than things you do, you know?

On-premises--yeah, sure, use k8s, it's literally the only place where it makes any sense to do so. Of course, you could write good code with clear interfaces and separation of concerns so you can figure out an on-prem story after you have a product and after you have a business, but again, I too like rabbit-holing on stuff that doesn't help me ship. ¯\_(ツ)_/¯ Good luck, I guess.

[arthurcolle]

> Training on the cloud is 20X more expensive, and you cannot use commodity GPU (banned by Nivdia).

Can you expand on this? What do you mean using commodity GPUs for training on the cloud is banned by Nvidia?

[streetcat1]

https://www.reddit.com/r/MachineLearning/comments/7ly5gi/new...

[arthurcolle]

Can any person selling a product essentially make unilateral decisions with respect to usage of the product in question?

Its very curious. I'm reminded of about 6 years back when I sent an email to Randall Stephenson (AT&T CEO) asking why they feel they can charge extra for tethering after selling "unlimited data." I included a parable, naturally, describing a baker who sells you bread, with a license agreement stating you can only eat the bread by itself. If you want to make yourself a sandwich you need to pay an extra 16% fee when you buy the bread to be able to use it for any derivative product, such as sandwiches or bread pudding, or croutons, or anything beyond raw bread.

This is nonsense. I own the things I buy and even if the law tells me otherwise, I will never accept these insane premises.