[stormbrew]

So I'm mostly curious why this isn't just basically serf 2.0. Looking at serf I never really felt like it had much use in the basic form it took, with no ability to advertise extra details about the nodes in a dynamic fashion. Consul seems to build onto serf the things that serf needed to become really useful, so seems more like a successor to serf than a parallel project.

It seems like the right thing to do here would be to take the lessons of building consul into making serf something more like a library on which to build other things rather than a service in its own right.

[mitchellh]

To add to what Armon said: we always had Consul in mind when building Serf. But Serf was a necessary building block on the way to Consul: highly available, lightweight membership management. Every distributed system has a membership problem, and we didn't want to reinvent the wheel for every system we built. Serf provided an incredibly stable and well proven foundation for us to build Consul on top, as well as future products that are already in the works...

And while you may not see Serf as having much use, we've personally helped and seen Serf clusters with many thousands of nodes. Serf is very useful to these organizations for its purpose. And while some of these orgs are now looking at Consul, many don't need Consul in the same way (but may deploy it separately).

We're not stopping with Consul. We have something more on the way. But we now have some great building blocks and experience building distributed systems to keep doing it correctly without having to rebuild everything from scratch.

[stormbrew]

It would be pretty awesome if you could get some case studies out about these deployments of serf. I would really like to hear more about them.

[armon]

The two offer very different trade offs and have different architectures. Serf is an AP system, meaning it trades of consistency for availability. 90% of your nodes can be down and a Serf cluster will continue to operate. It does offer a primitive set of features, but it is useful for a wide variety of tasks such as configuring a memcache pool, load balancers, P2P VPN topologies, etc.

Consul is a CP system, meaning it trades availability for consistency. It has a much more limited ability to tolerate failures. However, its more central architecture allows it to support a richer feature set.

By keeping the tools separate we give developers and operators two different tools. Sometimes you need a hammer, and sometimes a screwdriver will do.

This page compares the two: http://www.consul.io/intro/vs/serf.html