Even in a physical hardware, on-premise scenario, it's still easier to scale horizontally than vertically in almost all cases, for all the reasons I mentioned. That's a big reason why Kubernetes was adopted at an unprecedented pace at medium to large organizations - because it helps manage that approach.
They could have chosen Mesos instead. Kubernetes had other characteristics that allowed it to be adopted far and wide besides the ability to scale horizontally.
Besides, Mesos wasn't a good alternative for most companies, so saying "they could have chosen it instead" is a bit theoretical. Mesos was ambitious, but that made it less suitable for a plug & play system that fit easily into existing corporate systems, which had already adopted containers heavily.[] Another reason for Kubernetes' popularity is it didn't try to be a big leap forward the way Mesos did.
[]The Marathon container support for Mesos was released about a year after Kubernetes, but if you were going to set up a system for distributed orchestration of containers, it didn't make much sense to bring Mesos along for the ride. There's a reason Mesos is in Apache heaven now (the Attic.)
That is not very insightful. Your thesis started with the idea of elastic horizontal scaling.
Mesos was designed from ideas with HPC and catered to that. Large hardware capex, which is not elastic. Containers did not make sense in that world when it was architected, and it was retrofited into Mesos's architectural foundation.
Kubernetes was designed for a wide variety of workloads, and designed to be composable, versatile, and extensible. It has a far more decentralized approach, an antithesis to Mesos's Data Center as an OS approach. It isn't that Kubernetes did not try to do too much. It's that they laid a much more flexible foundation.
It turns out, Kubernetes was a better fit for a many more use-cases, even beyond large and medium sized enterprises. Kubernetes works pretty well at the edge and locally as well, and it runs many ML workloads on the other end of scale.
Your version of the story is fine, but the point remains, Mesos was not a good fit for most medium to large organizations at the time they began adopting Kubernetes. As such, those organizations rationally did not "choose Mesos instead."
that's..kind of not true. they weren't elastic in the sense that you never had to think about how big they were. but you had say 64k nodes, and people would launch jobs with 1000 of them, or 10000, or if if they could clear the decks all of them. or if they were just debugging, maybe 5 of them.