[dnomad]

Exactly. Docker files are worse than war files in pretty much every way.

The economics are interesting though. Computing resources are super cheap and getting more cheap everyday. (Well except for RAM!) The waste produced by containers means says, yeah, give each application its own app server. Its own web server, its own JDBC driver, even its own JDK. We'll throw it all on the cloud and run it for a few cents each hour.

The problem is that testing isn't cheap. This is where microservices and containers and the all this craziness will fall down. Now you've got dozens and dozens of applications all running with their own application servers, JDBC drivers, and their own JDK. This is a combinatoric explosion in your testing surface.

The true value of the application server approach is that it forced applications to conform to a clear contract. Once your application conformed to this contract it could be thrown over the wall into a full-time ops organization that could transparently deploy, monitor and manage the app.

The container madness will come to an end. People will realize that giving developers the keys to the kingdom and total freedom over their application is a terrible idea. (And smart developers don't want total freedom.) What will emerge will be something much more interesting: a hybrid approach where applications can be bundled into into artifacts that fully express their dependencies and containers can be annotated in a way that fully express their capabilities.

Containers provide isolation and immutable infrastructure and this is good. Appservers provide standardization, specialization and separation of responsibilities and this is also good. There's no reason why we can't enjoy both.

[tyldum]

We use containers to reduce the testing surface and enforce standards. And if course to ease testing as we now transition the same build through dev, staging and prod.

The devs just have a single base container to work off from and we use CICD to further enforce whatever we need to.

It also makes it easier for devs to propose changes, as we know we can consistently apply them.

[haskellandchill]

You can use them to do that but is it working? That was the point, I also agree that no, it is not working and has rather introduced several layers with painful and hard to debug failure modes.

[taeric]

A problem comes down to answering questions such as "exactly what versions of X are installed in our beta stack?" Usually trivial if X is part of your application. Much more complicated if X is some lib that is on the system/image.

[mmt]

> People will realize that giving developers the keys to the kingdom and total freedom over their application is a terrible idea. (And smart developers don't want total freedom.)

Sadly, I don't think it's actually developers that were pushing for this "freedom" (with which comes responsibility, hence it makes sense that it would be unwanted), as such.

Rather, I place the blame with managers and with (fellow) sysadmins for not embracing, empowering, or forcing (whatever it takes) the aspects of the original DevOps cultural movement to have Ops bring the equivalent [1] of a production environment to every developer.

> Containers provide isolation and immutable infrastructure

I thought they weren't even (necessarily) immutable. If not, then maybe they're merely immutable-through-practice, but that's as unsatisfying as configuration management systems creating reproducible deployments that lack traceability [2].

[1] In terms of tooling for things like builds, deployment, and management (especially of dependencies), not necessarily full hardware capacity, although at one place I did provide each developer with a copy of yesterday's production database on their own, personal bare-metal database server, less "beefy" than the production hardware.

[2] is that the right word? something like chain-of-custody for code and/or configuration.

[kinghajj]

Nothing prevents one from having a container with many WAR files, so that there's only a single JVM for multiple applications.

[dnomad]

The entire exercise is about expressing dependencies. (The real third hard problem in computer science. Only idiots fall for off-by-1 errors.)

Containers are one way to force an application to explicitly declare all of its dependencies. The problem is the application then also bundles those dependencies. There's not much room for control or intermediation. Developers have total freedom to do all sorts of wackiness. (And oh the wackiness I've seen in containers. When the cat is away...) What's needed is a richer (and much more compact) mechanism for applications to express all their dependencies.

Another way to understand the issue to understand the underlying economics at work here. Container economics proposes that computing resources are so cheap that, sure, let the developers go crazy and do whatever they want. But those same developers must be on the hook for all operational concerns hence "DevOps." And this will scale until one day you look around and you realize that anarchy never works, not even a little bit.

Now there's a lot of room to fall here. RedHat is in the business of selling compute and storage resources. It's not clear that even if they could make their products more efficient they would. But as long as compute resources are cheap and getting cheaper you can let people go wild and RedHat will make a lot of money and everybody will be happy. For an organization the true costs will emerge in a very subtle manner: the productivity of the developers (already hard to measure) will ultimately fall. That's because instead of actually solving actual business problems their devs are deploying dozens and dozens of polyglot microservices and trying to figure out why on earth microservice #28 that Bob, who left the company a year ago, decided to write in a version of lisp that he invented crashes every day at noon.

But hey, at least Kubernetes makes it easy to deploy everything!

[parasubvert]

> There's not much room for control or intermediation.

FWIW there are container-based app platforms that do allow you to swap out filesystem layers to update dependencies and to remove control from developers by having a standardized containerizer that has extension hooks but can't be mucked with at the lowest levels. This is how Cloud Foundry works for example, or Heroku.

[smarterclayton]

For example, OpenShift (Kubernetes + stuff also from Red Hat) exposes this pattern like both heroku and cloud foundry by

A) focusing on having standard base images controlled by ops

B) encouraging combining source code / built artifacts as a layer on those base images

C) giving controls to ops so that the only images users could build or run must be built with A/B above.

In that mode containers are less wasteful because you can share the base image across every host (or rebuild everything centrally), and all that gets downloaded to a host is the source code top layer. Which is roughly indistinguishable from the lambda runtime and how it accesses the code to execute.

[taeric]

Don't most virtualized hosts already solve the "less wasteful" point? I get that hypervisors and friends aren't bulletproof, either; but I have fewer ways to shoot yourself with those than I have with containers.

More, you have to worry about the container host anyway, so you haven't removed the need to maintain a host. You've just added to it the need to maintain the rest of the container infrastructure as well as your application stuff.

For places that have not solved the "virtual hosts should be virtually free" problem, containers are quite welcome. You can get going with them quite quickly. If you have already solved that problem, they can look an awful lot like just more work.

[smarterclayton]

Agree, virtual hosts are ubiquitous. But I don’t know that anyone loves their virtual machines the same way they love the smaller, faster, and simpler alternatives (unikernels included).

This is a somewhat pessimistic viewpoint, but lowest common denominator solutions tend to acquire the most network effects. A VM requires more touch points to manage for the person who has to set up a machine - despite ten years of solid progress, they still tend to be pretty annoying to configure and build and manage. The platform as a service approach (whether lambda, nodejs on cloudflare, various functions as a service approaches, heroku, cloud foundry, or dokku) on the other hand take away a lot more hassle by abstracting pain points out, but get accused of being too rigid. Both extremes benefit specific use cases, but have disadvantages in general purpose use.

Containers sit in the ugly, dirty, practical middle. They can do both (VMs are just processes). So the network effects they accrue just like Linux did of being “good for everything, not great” help mitigate some of the disadvantages.

The public cloud providers change this calculus a bit by offering these things as a service, but internally they are just managing the container runtimes for you.

I’m obviously biased, but I tend to see containers as “good enough” to build other abstractions on top, with specific areas where VMs and heavy PaaS abstractions clearly win.

[taeric]

To be fair, it is more likely that random microservice #28 is running fine, but folks don't like it and are uncomfortable modifying it to add a variation to one of its rules. Thus giving birth to microservice #28'. :) (Please read this tongue in cheek.)

[parasubvert]

> Exactly. Docker files are worse than war files in pretty much every way.

They focus at far too low level a problem.

> The problem is that testing isn't cheap. This is where microservices and containers and the all this craziness will fall down. Now you've got dozens and dozens of applications all running with their own application servers, JDBC drivers, and their own JDK. This is a combinatoric explosion in your testing surface.

I fail to see how this more difficult from 95% of existing enterprise systems that run various JDK versions, various WebSphere versions, JDBC drivers etc.

The difference with containers is that vendoring your dependencies makes all that explicit. Whether you choose to stay on old versions is a mistake you can still make (and will make if you leave container creation and patching in the hands of dev teams).

That said, since when does any organization build horizontal testing libraries, like, "test X version of JDBC", so they can standardize on a single driver across all projects? It's never done. Projects are all over the place with their dependencies unless there is institutionalized forcing through Maven repos or what not to block the download of old/insecure versions.

There is no combinatorial problem here, each test suite needs to unit test the individual service and then test the API contract of the service. If you let your dependencies decay you're accepting major security and maintenance risk, but you were in the WAR file world too.

> The true value of the application server approach is that it forced applications to conform to a clear contract. Once your application conformed to this contract it could be thrown over the wall into a full-time ops organization that could transparently deploy, monitor and manage the app.

LOL, I really think you're exaggerating. I've worked with WebLogic, WebSphere, IIS, JBoss, Tomcat, Django, Rails, Node, you name it over the years, and while some orgs got close in the Java world, this was mostly a pipe dream that never happened in any cost effective manner as a standard practice.

> The container madness will come to an end. People will realize that giving developers the keys to the kingdom and total freedom over their application is a terrible idea. (And smart developers don't want total freedom.)

This I agree with. It's all far too low level.

This really was Docker, Inc.'s fault IMO, their marketing message was to empower developers to get ops out of the way by building these containers that would magically just be run as opaque lego blocks. That was the hype everyone on HN was drooling over in 2013. Turns out it's not quite that simple.