[bruinjoe]

I see a lot of value in Kubernetes. For one thing I don't want to wake up at 1 AM if one of my node goes down and brings down with it important containers. If I scaled the environment with excess capacity then Kubernetes should start up new pods on the remaining nodes.

[tnolet]

That is an awesome feature of K8s and similar orchestators. It is however just an HA (high availabilty) problem already solved many times over in many ways. Almost every PaaS system has it. All cloud providers have it. Getting K8s (or any of its competitors) for HA is total overkill.

[manigandham]

We use a GKE cluster with dozens of preemptible/spot instances. Great savings for no extra management fee, and with a nice standard kubectl interface for deployments and changes.

VMs are the most reliable thing of any cloud, and often more reliable then their own managed services because of complexity. GKE does live-migration of VMs and Azure/AWS are close to the same thing, so building on that foundation and doing your own "managed" layer via Kubernetes provides a better experience, while also giving you that spot instance discount that you wouldn't get otherwise with PaaS.

[kuschku]

The point is that I can rent the cheapest $10/month servers I can find, a bunch of them, throw kubernetes on that, and it’ll still be up.

Same reliability as AWS, similar performance, much cheaper.

[yongjik]

Unless you're renting from multiple $10/month services, your service can be only as reliable as your service provider, which is probably less than AWS.

[rhizome]

Doesn't the same condition apply to AWS, only they're called "availability zones?"

[bruinjoe]

You hit it on the head. Weave Net is participating in the Istio project. I hope they can bring WeaveNet to Istio. That might be able to provide a multi cloud vendor service mesh.

[kuschku]

But that’s easily possible – you can use all the French hosters at once. Latency between them is practically nil because their data centers are only a few hundred meters away from each other.

Sure, in case all of Paris goes down, you’re down, too. But in day-to-day usage you can have good reliability on a shoestring budget.

[int0x80]

What good $10/month servers do you know?

Genuinely interested.

[firebacon]

Nobody wants to wake up at 1AM because their singly-homed service just went down. Kubernetes might be a fine tool to achieve that, but I want to point out that there are much simpler failover solutions available. Failover is something you should definitely have, but also something you definitely don't need kubernetes for.

[zzzcpan]

In fact, kubernetes won't solve high-availability for you. It is completely orthogonal to failover.

EDIT: No, really, k8s won't make a typical RDBMS suddenly able to run in three datacenters across three continents.

[pests]

But it can. You can develop a custom controller that does manage a typical RDBMS to be able to run in three datacenters across three continents.

[pritambaral]

> You can develop a custom controller that does manage a typical RDBMS to be able to run in three datacenters across three continents.

Honestly, at that point, what does k8s get you?

AIUI, Kubernetes is fine for stateless systems, but it is really no better on the storing-state question.

[wmf]

It's easier to install Stolon under K8s than by hand. :-)

[pests]

Consistency.

[zzzcpan]

It was a trick sentence, RDBMSs that can do that and remain useful enough for at least some applications don't exist.

[shaklee3]

Huh? One of the core features is making sure you have N pods running at all times. If one fails, it starts another.

[zzzcpan]

This feature won't even work for anything designed for fault tolerance. I believe it can only work for things that either rely on other properly fault tolerant services or that are completely stateless, idempotent, so they can be retried and eventually consistent (in which case the state still has to be handled by properly fault tolerant systems). Either way kubernetes cannot help here.

[pacala]

Name one.

[firebacon]

You mean a failover solution? It's hard to give a list here because it is such a large space and it completely depends on your product/application. It is more like a category of use cases than one specific use case.

Some ideas for things you could do in a web/website/internet context assuming you have a single point of presence:

One type of "HA" is network-level failover; haproxy (L7), nginx (L7) and pacemaker (usually L3) seem to be very popular options, but I think there are dozens of other alternatives. In terms of network-level failover, things get more interesting once you are running in multiple locations, have more than one physical uplink to the internet or do the routing for your IP block yourself.

For application-level failover and HA, one option for client/server style applications is to move all state into a database which has some form of HA/failover built in (so pretty much every DB). I think this is very common for web applications and also for some intranet applications.

Assuming you have a more complex application running in a datacenter, there is also a lot of very interesting stuff you can do using a lock service like Zookeeper or etcd or really almost any database. Of course, you can also make your app highly available without using an external service; there is a mountain of failover/HA research and algorithms that you can put directly into your app (2pc, paxos, raft, etc). Of course all of these require some cooperation/knowledge from the application. For some apps it might be very hard to make them "HA" without relying on an external system, but for some apps it will be trivial.

Note that when you move away from a web/datacenter context, to something like telecommunication or industrial automation, so something that doesn't run as a process in a datacenter but is implemented {as, on} hardware in the field, failover and high availability will have an entirely different meaning and will be done in a totally different way.

[peterwwillis]

Most single-instance, single-zone failover scenarios can be handled with shell scripts, the AWS API, and cron.

But the parent's comment is missing the point. K8s is not for failover. K8s is literally just a giant monolith of microservices for running microservices. It's not intended to provide failover for non-microservices, it's intended only to run microservices, and as a side-effect of needing to be able to scale them, it inherently provides some failover mechanisms.

[pacala]

Given GKE or AKS, not quite sure how "shell scripts, the AWS API, and cron" is "much simpler" than:

   $ docker build . -tag gcr.io/google-samples/hello-app:1.0
   $ docker push gcr.io/google-samples/hello-app:1.0
   $ kubectl run hello-server --image gcr.io/google-samples/hello-app:1.0 --port 8080
   $ kubectl expose deployment hello-server --type "LoadBalancer"

where the Dockerfile content is:

   FROM golang:1.8-alpine
   ADD . /go/src/hello-app
   RUN go install hello-app

   FROM alpine:latest
   COPY --from=0 /go/bin/hello-app .
   ENV PORT 8080
   CMD ["./hello-app"]

https://cloud.google.com/kubernetes-engine/docs/quickstart

https://github.com/GoogleCloudPlatform/kubernetes-engine-sam...

[firebacon]

The difference in simplicity is not in the interface that is presented to you as a user. The difference is that your shell script will have a couple hundred lines of code, while the docker and kubectl commands from above will pull in literally hundreds of thousands of lines of additional code (and therefore complexity) into your system.

I'm not saying that is a bad thing by itself, but there definitely is huge amount of added complexity behind the scenes.

[tannhaeuser]

That's nice and all as long as it works. If there are any problems with it (network, auth, whatever) have fun with even diagnosing the bottomless complexity behind your innocous lil' kubectl command.

[pacala]

The error messages are quite good. There are extensive logs. There are plenty of probes in the system. Just the other day we `kubectl exec <pod_name> /bin/bash` to test network connectivity for a new service.

To the best of my exposure, Kubernetes is a well engineered system.

[aphexairlines]

CodeDeploy has been doing this for ages.

https://docs.aws.amazon.com/codedeploy/latest/userguide/gett...

Distelli (now Puppet Pipelines) too:

https://www.youtube.com/watch?v=ZZlYADohS4Q

And then there are the PaaS options like Heroku, Beanstalk, AppEngine, Lambda, Serverless, and Apex.

[firebacon]

> But the parent's comment is missing the point

That was my point. I wanted to point out that while some people have only/first heard about failover in the context of kubernetes, it is not something that is specific to kubernetes or even the problem that kubernetes was build to solve.

Of course it is not designed to be a failover solution specifically and using it (exclusively) as such would be ill-advised; I was just trying to be diplomatic while pointing that out.

[jontro]

We're using elastic beanstalk currently (multi docker conatiner). Not that I'm advocating it and I'm really interested in k8s now that aws has eks, but ebs is really simple to use for a simple setup.