Hacker News new | ask | show | jobs
by ryanl-ee 3738 days ago
Have you checked out DCOS? Once you have your cluster set up, your path to installing Spark is running "dcos package install spark" on your laptop.
1 comments
minimaxir 3738 days ago
Huh, no I haven't. Although the package support appears to be more narrow than with Kubernetes/Docker, but DCOS is a possibility too.
link
ryanl-ee 3737 days ago
Well, the included packages in the "dcos package" CLI repo are just the prepackaged frameworks. You can actually deploy Kubernetes on top of DCOS in the same way - it natively supports Docker containers as an execution method.
link
ptrincr 3737 days ago
You should also check out Ambari, fantastic open source Hadoop/spark cluster management tool.
link
mindcrime 3737 days ago
Second the motion. Ambari is actually one of the key elements of the tech stack we're using for our service. With the "Blueprints" feature and the REST API, it's a really convenient way to automate the provisioning of Spark/Hadoop clusters.

Even better, Ambari isn't actually limited to installing just Hadoop/Spark, etc. In principle, you could extend it to take care of installing pretty much anything.

link