[jacooper]

What about Nomad?, I hear it isn't as complex as K8s, but still offers the same capabilities.

Also in what way is Swarm is abandoned?, I mean if it works fine, and is still supported in Docker-CE than its still OK to use it, at least in small businesses and hobbyist use cases where Swarm's simplicity are attractive.

[techdragon]

I recently looked at Nomad, but ironically for a "less complex" piece of software, I couldn't be sure it would scale down to my use case, I ended up with K3S as it would run on the small hardware nodes i needed it to run on.

I would have preferred Nomad but the resource requirements are pretty high for the "control server" component. https://www.nomadproject.io/docs/install/production/requirem...

  Nomad servers may need to be run on large machine instances. 
  We suggest having between 4-8+ cores, 16-32 GB+ of memory,
  40-80 GB+ of fast disk and significant network bandwidth. 
  The core count and network recommendations are to ensure
  high throughput as Nomad heavily relies on network communication
  and as the Servers are managing all the nodes in the region and
  performing scheduling. The memory and disk requirements are due to
  the fact that Nomad stores all state in memory and will store two
  snapshots of this data onto disk, which causes high IO in busy
  clusters with lots of writes.

Obviously This is not going to fit on a group of Raspberry Pi's or other SBC computer nodes you can solar power out in a field.

[cb22]

I too was initially put off by those requirements. Now we run our Nomad server on a single t3.medium instance that sits at a 0.1 15 minute load average and has ~500MB of RAM used.

This manages about 100 client nodes. No need for a cluster since we don't need high availability on our control plane, and there's no actual state stored there that isn't created from our CI pipeline.

[schmichael]

Glad you went for it despite our poor documentation cb22! That sounds like a great setup. I think the only defensible way to describe our "Requirements" page is that we wanted to make the safest suggestion for the widest range of users. Obviously it's wildly inaccurate for a wide range use cases, and we should fix that.

Nomad servers could start with 300mhz, 100mb of memory, and eMMC storage and run a RaspberryPI cluster just fine. Our most important resource guidance is all the way over in our Monitoring docs!

> Nomad servers' memory, CPU, disk, and network usage all scales linearly with cluster size and scheduling throughput.

https://developer.hashicorp.com/nomad/docs/operations/monito...

Any cluster can start with 300mhz and 100mb as long as they monitor usage and scale appropriately.

We're going to try to update our Requirements docs to add this nuance and guidance on how to calculate requirements based on projected load. We recently spent some time improving our heartbeat docs, and I think the approach we took there will serve us well for system requirements: https://developer.hashicorp.com/nomad/docs/configuration/ser...

[techdragon]

Thank you for the follow up! This makes it much more likely that I’ll remember to give it a shot when the next applicable project comes up… Because I’ll probably have forgotten these comments and gone to check the docs again :-)

[techdragon]

See this is the sort of information that they should have posted on the requirements page. Not some arbitrary sized high water mark needing gigs of disk and memory. I'll have to give using Nomad (and Consul) another go next time, and test it on physical hardware to see.

Thanks for the info! and for doing what Hashicorp seemed too busy to do themselves.