Hacker News new | ask | show | jobs
by domenp 4096 days ago
Allow me take the opportunity and ask if anybody here is running Elasticsearch with Mesos (and Marathon) using Docker container?

I'm running Elasticsearch nodes on a dedicated Mesos slaves and I’m still not sure how much memory should I allocate to Elasticsearch task e.g. all available memory as reported by Mesos or some smaller amount to leave something for the system? Please note that I'm asking about memory allocated to the Marathon task not about JVM heap size.
1 comments
nemothekid 4096 days ago
I've taken the view of treating our database slaves as special and allocating all the memory on the slave to ES. That way ES (or any other db) always runs on the right slave, and no other potentially long running/resource hungry process runs alongside it (but just enough that's jenkins tasks and chronos tasks can still rune if need be).

AKAIK, there isn't a right way to run databases because the persistent storage layer into mesos is still being baked.

link
dingdingdang 4096 days ago
> the persistent storage layer into mesos is still being baked

That's pretty important point, would have thought persistent storage would have been first point to get working for a project like Mesos. Also, their homepage ( http://mesos.apache.org/ ) outright states: "Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines [...]". What storage are they talking about if not persistent storage?

link
nemothekid 4096 days ago
This mesos 0.22 talk goes into a bit more what they are imagining - http://mesosphere.com/2015/03/27/mesos-0-22-0-released/

Essentially the current mesos disk quotas are just so that tasks that run, run with a minimum amount of disk space. I think what they are trying to accomplish in future releases is some way you could build an EBS Style disk space management on top of mesos.

link
SEJeff 4096 days ago
Yes, they are calling it dynamic reservations:

https://issues.apache.org/jira/browse/MESOS-2018

With a first draft of the user documentation at:

https://gist.github.com/mpark/e8ee4eb9671bdb252c4f

It will be really slick once this makes it all into Mesos 0.23

link