The Mesosphere DCOS is built around the Apache Mesos kernel
The Mesos kernel was developed at UC Berkeley in 2009 [1].
Spark was written as a sample app on top of it [2].
Ben Hindman and his colleagues at the UC Berkeley AmpLab had always envisioned Mesos as a kernel inside of a full-blown operating system [3]. They finally brought it to market.
[2] "We have implemented Mesos in 10,000 lines of C++. The system scales to 50,000 (emulated) nodes and uses ZooKeeper for fault tolerance. To evaluate Mesos, we have ported three cluster computing systems to run over it: Hadoop, MPI, and the Torque batch scheduler. To validate our hypothesis that specialized frameworks provide value over general ones, we have also built a new framework on top of Mesos called Spark, optimized for iterative jobs where a dataset is reused in many parallel operations, and shown that Spark can outperform Hadoop by 10x in iterative machine learning workloads." ibid.
My view on this the following: Mesos is similar to the kernel of a conventional operating system (e.g. Linux). It provides very basic services (scheduling, interrupts, device management, etc) and a syscall API. But nobody wants to program to this API. Hence you need libc or other similar libraries to provide a higher level API that programmers use to interact with the kernel. Kubernetes, Marathon, Aurora, etc are such libraries, each optimizing for a different class of applications and providing different functionality. The two (the kernel and the libraries) need each other.
I see Kubernetes as more of a programming model, not an operating system. It provides a way to express services and have them scheduled onto a datacenter. The rocket-science of how you schedule those workloads and optimize them in the same partition as other workloads is what you need a DCOS for.
In the Mesosphere world, Kubernetes is a "datacenter service" which is installed on your datacenter so that you can run Kubernetes workloads. You might also want to install DEIS to run DEIS-organized workloads. Or Spark, for Spark workloads... and so on -- all multitenant in the cluster. This is what the DCOS is uniquely good at, and why it qualifies as a true operating system.
The Mesosphere DCOS is built around the Apache Mesos kernel
The Mesos kernel was developed at UC Berkeley in 2009 [1].
Spark was written as a sample app on top of it [2].
Ben Hindman and his colleagues at the UC Berkeley AmpLab had always envisioned Mesos as a kernel inside of a full-blown operating system [3]. They finally brought it to market.
[1] https://www.usenix.org/legacy/event/nsdi11/tech/full_papers/...
[2] "We have implemented Mesos in 10,000 lines of C++. The system scales to 50,000 (emulated) nodes and uses ZooKeeper for fault tolerance. To evaluate Mesos, we have ported three cluster computing systems to run over it: Hadoop, MPI, and the Torque batch scheduler. To validate our hypothesis that specialized frameworks provide value over general ones, we have also built a new framework on top of Mesos called Spark, optimized for iterative jobs where a dataset is reused in many parallel operations, and shown that Spark can outperform Hadoop by 10x in iterative machine learning workloads." ibid.
[3] http://people.csail.mit.edu/matei/papers/2011/hotcloud_datac...