[dkhenry]

I would say Kubernetes is becoming the new Application Server, but not for anything listed in the article. Kubernetes is more and more being sold to "enterprise" as a solution for running services much like Application servers were, and much like application servers the standard Kubernetes deployment is becoming a tangled mess of buzzwords and dreams, which work great in a demo, but won't work at all without an army of consultants.

[brozaman]

I'm a former OpenShift (Red Hat's distribution of Kubernetes) consultant and currently I work at Red Hat in a different position and I have to disagree on the "won't work at all without an army of consultants".

While it's true that some of our customers have an army of consultants, the vast majority of our customers don't use consulting at all or that only use it very infrequently. If you don't want a lot of customization, have the right amount of people and the right people and realistic expectations you don't need consulting at all.

When people want to get a highly customized experience (often, for the wrong reasons), or want to get in production after 2 months, when their people have no experience in kubernetes and didn't do adequate testing (load, fault-tolerance, etc) it will give lot of problems. But that's the case for everything, not only for kubernetes

In my opinion (mine, not Red Hat's) getting someone to deploy it with you the first time, showing how it's done and why things are done, and after that and do a workshop explaining the basic concepts has great value, saves a lot of time and isn't expensive.

[dkhenry]

This is the same process that was done with J2EE and Application Servers, it starts out with a _simple_ standard to have a Java application served with a standard API, and then it grows out of control. You are already seeing it with Itsio and Helm being layered on top. Today you might be able to get the stock Kubernetes up without much assistance, but give it a few years as "features" get added on.

[brozaman]

I'm not seeing that and I'm pretty sure we won't. it's extremely hard to get new features in the core of kubernetes and in fact it's moving to a micro services architecture.

Neither helm or istio are part of the core.

[kiallmacinnes]

I think you're making a distinction that many others wont. Specifically, a distinction that ordinary users of Kubernetes won't make.

Kubernetes is more than what's in the Kubernetes core GitHub repo, it's the collection of tools that go along with Kubernetes that "average users" expect to just be there. For example, an ingress controller, helm, likely soon cert-manager and external-dns too.

The split out into multiple projects actually makes the landscape harder. Where by people and distributions have to make choices about the set of expected vs set of provided features.

A great example of this I saw was an internal thread here at SUSE (I work on their Kubernetes product) where ingress controllers were being compared. Despite the Ingress data model actually being in the core, features like nginx-ingress-controllers ability to do arbitrary TCP and UDP ports on the same external-IP were being seen as a reason to choose that controller over some of the others. I can't argue the feature is useless, because it's not - it's very useful - but it's a great example of how things can very easily become conflated, even for people who should know better! A feature being part of the core just means it's part of a minimal Kubernetes deployment, while a complete Kubernetes deployment includes much more than core.

[mmt]

> want to get in production after 2 months [...] > it will give lot of problems. But that's the case for everything, not only for kubernetes

Are you asserting that any modern production environment that takes less than 2 months to set up is bound to cause problems? Or the whole company/startup/project?

If the former, I'd say there are many tech startups who would disagree, or at the very least, point out that it doesn't matter because they'd cease to exist during those 2 months without an MVP in production.

[marenkay]

I think the two month phase is pretty accurate. Deployment itself is not that time intense and changing your applications to be container friendly is not either.

The time intensive phase during the first Kubernetes deployment is changing the mindset of the engineering team and everyone involved in IT.

Kubernetes IMHO is pretty much like moving from college into work life and adapting to the fact that requirements for a decent adult life are very much different from a college life.

[mmt]

That's as may be for Kubernetes, but the GP specifically stated "that's the case for everything, not only for kubernetes", which struck me as an extraordinarily broad claim.

One might reasonably expect "everything" to include traditional deployment methods that require no changing of mindsets (not that that's likely a factor in my nascent startup example).

[marenkay]

Transition phases actually make sense even if you apply "everything" instead of just "Kubernetes" considering how every company transitions through stages in terms of tooling.

Think being a startup where deployments most likely will be manual and undocumented, upgrading to an initial automation using some scripting, transitioning to CI/CD, etc.

So from various experiences at companies of all sizes I would actually support such a broad claim, since companies are basically transitioning all the time through new phases to greener pastures. That of course assumes that the company values constant improvements and uses an iterative process for growth and change.

[brozaman]

I should have worded that differently. I meant migrating an existing production.

If you have an existing production which works well after years of improvement and stabilization, with people with a lot of expertise on it, you won't get that in 2 months, not the stability and quality nor the ability to troubleshoot it.

Startups start from scratch, don't have big workloads, also there isn't so much pressure to have a high quality service from day one because you don't have that much people consuming your services.

[mmt]

Thanks. Where "everything" is in the context of a transition or migration, rather than any kind of deployment, including an initial one, 2 months seems like a very reasonable minimum.

It's certainly in line with technical new hire productivity "ramp up" time estimates (typically 3 months?), which I would guess are a similar (if individual) case of change-of-mindset.

[colechristensen]

Very similar abstractions are being created over and over, if you are too close it is hard to see that the abstractions are largely the same, and the grand benefits being toted are actually quite small when comparing one to the other and the differences are just attitudes and a tiny bit of tooling that could be recreated for any of the abstractions.

[taeric]

It does get frustrating to lose tooling you previously had. Even more frustrating when some of the instrumentation that you take for granted in the old world is "TBD" in the new one.

[threeseed]

Instrumentation on Kubernetes is pretty incredible albeit complex. You have host, container, application and ingress level metrics. But via the mesh networking eg LinkerD detailed metrics of how containers are talking to each other.

All of which roll into a single metrics server (Prometheus) and dashboard (Grafana).

[taeric]

Most of the instrumentation I have found myself losing have been at the application levels. And to be clear, most of them are still somewhere, but not nearly as well polished as in services we have not on containers. (I'm also primarily talking about a bunch of internal practices at my current job.)

[threeseed]

I can't speak to how you build your apps.

But application level metrics for example on the JVM behaves exactly the same whether it's in a container or not. You can still pull JMX metrics.

[taeric]

Agreed, most of this is in the "how you build your apps."

Setting up all of the special networking tricks to get to the bloody JVM in a container can be annoying. Especially because I guarantee those weren't setup until they were needed.

But yeah, most of my complaint is with folks that dropped the old way of making applications because they wanted to use the new tooling. Completely missing all of the "hidden" use cases the existing applications were made with.

[dsmithatx]

I'm helping build an Openshift platform with 2-3 redhat consultants. I do not work for Redhat, I'm a Linux/Devops admin. It's been very simple to install and fully automate everything. It also is allowing developers to focus on writing code and not infrastructure as AWS does. Most of it is bare metal for now, but we will be able to spin up AWS and Azure in a few hours with the same code.

The instructions are accurate, it's been easy to learn, and does not require an army of consultants. We will be moving 1000's of apps to multiple data centers globally with a team of around 6 people. It took 100s of employees years to build and maintain infrastructure for those apps.

Note - Consultants are not needed at all, they simply help speed up the process with expert knowledge.

[marenkay]

I have to disagree here. From personal experience I can say: you need exactly one person willing to read, try and properly observe the installation and population of a kubernetes cluster with applications and tools.

The actual cluster deployment is not mess at all. Even if done with a mere Bash script, it can be done in like 20 lines of code.

I can only recommend to actually try using Kubernetes because what you wrote reads more like you are actually still in the state of "wtf is this and why should I touch this crap?" but have not yet tried it.

[threeseed]

I can spin up a Kubernetes cluster in seconds using AWS, Azure or GCP. And with helm installing new applications is a pretty trivial affair.

I am not disagreeing that it’s complex and a lot of buzzwords but every day the situation is getting better and better.

[dkhenry]

You can also spin up Tomcat with everything needed to run an application with your local package manager and you can deploy your WAR file by putting it in the right location, but that's not what IBM, Oracle, and Red Hat have sold as Application Servers.

This is effectively what is being sold now but using Kubernetes and Containers instead of Application Servers and J2EE

[derefr]

The difference is that Tomcat can’t run arbitrary badly-behaved third-party x86 binaries under itself. (I mean, it can, but it doesn’t have the facilities to protect itself from them.)

K8s lets you build “containers” out of whatever existing software happens to be laying around, rather than having to write your services from the ground up for being run in a container.

[halbritt]

Easy peasy.

Now try troubleshooting it.

[threeseed]

On the cloud providers it is fully managed so nothing really to troubleshoot from the core platform perspective. The applications e.g. ingress will likely do but that's the same as if you were running on bare metal without Kubernetes.

And again the monitoring and auditing on Kubernetes is significantly better than trying to roll your own. There is detailed logs and metrics up and down the stack.

[halbritt]

"On the cloud providers it is fully managed so nothing really to troubleshoot"

Clearly, you haven't run much at scale or experienced many problems.

Troubleshoot a network issue through an ingress, kube-proxy and then down to the pod.

Troubleshoot an issue with kube-api or etcd when you have no access to either the master or the etcd cluster.

Troubleshoot a load issue on a node related to IO.