[dharmab]

Most famously, Siri (used to?) run on a very large scale Mesos deployment (10000s of nodes, much higher than Kubernetes can scale to).

Unfortunately the original article is lost, but here's a summary: https://daringfireball.net/linked/2015/04/29/siri-apache-mes...

[thamer]

Wayback Machine has it[1], but there's not much more content than in Gruber's summary.

[1] https://web.archive.org/web/20150429225603/https://mesospher...

[jeffbee]

OK but what was the utilization? I'm not really sure K8s is state-of-the-art either. There are published research papers about very-large-scale clusters with 80%+ resource utilization.

[dharmab]

In our production experience, utilization had far more to do with the service owners (or autoscalers/auto-tuners) correctly choosing the cgroups and CPU scheduler allocations, as well as the kernel settings for cgroup slicing and CPU scheduler. We had Mesos clusters with 3% utilization and have Kubernetes clusters with 95%+ utilization. But we also have Kubernetes clusters with <10% utilization.

[madhadron]

To be fair, Kubernetes right now only schedules relatively small clusters. But it turns out that the majority of the world is not Facebook or Google and only needs relatively small clusters.

[davidopp__]

> To be fair, Kubernetes right now only schedules relatively small clusters.

This is not really true:

https://news.ycombinator.com/item?id=25907312

https://cloud.google.com/blog/products/containers-kubernetes...

https://www.infoq.com/presentations/alibaba-kubernetes/

[dharmab]

Even those numbers (10k to 15k nodes and 100k containers) are smaller than what a great Mesos framework was capable of.

Of course, this mattered to only a very small number of organizations.

[madhadron]

Yes, 10k to 15k machines is a relatively small cluster in my world.