[movedx]

I've come from bare metal, to virtualisation, to Cloud. I then went from AutoScaling Groups and AMIs to K8s and Docker Images. Recently I went from K8s to Nomad+Consul and Docker Images.

Now I'm building a small platform to host a business I've started and I'm going back to ASGs with simple EC2 Instances.

There's really no need for the features Nomad/K8s offer at almost any scale outside of a few big companies.

[Axsuul]

It depends. Orchestrators like Nomad can give you a lot of leverage as a small team. It's not very difficult to manage and can give you lots of redundancy, control, and a path to scalability while utilizing best practices so you don't have to think about it. There are lots of ways to do this and not every app needs this but as you begin to get traction, you either have a choice of outsourcing your devops or bringing it in-house which happens way before you are even a big company. Nomad solves the problem of the latter.

Also, I'm curious why you didn't just end up sticking with Nomad now that you know it.

[movedx]

Orchestration engines can give you a lot of leverage if you run a managed solution because all you're doing there is deciding on the size you need and providing a credit card number. This is absolutely a great way of your DevOps to the next level with little effort.

If you're going DIY, however, then you're not doing yourself any favours by starting out with an orchestration engine.

In the case of Nomad you'll also need Consul for it to be truly useful. That means six EC2 Instances (for example) for a truly production trade system - 3x for Nomad, 3x for Consul. Of course you also need mTLS as well - may as well throw Vault into the mix too, then... that's another three EC2 Instances to support that cluster... 9x EC2 Instances into the DIY solution and all you have is an orchestration engine and you've not even started on the workers yet. That cost so far is USD$30,416.04 for the reference architecture for all nine nodes: https://learn.hashicorp.com/tutorials/consul/reference-archi...

On the other hand if you keep it simple and just work with AMIs you bake your self when you need to release something new, you need two EC2 Instances. The cost of the compute resources for this for 12 months: UDSD$330.

There's no raft consensus which needs three nodes to be stable (but a single AZ failure can result in split brain, so really you need five AZs), no new technologies to understand, no new vendor to work with or pay capital to, nothing more to support.

These things - these orchestration engines - have their place, and that place is outside of 99.9% of businesses (large or small.)

As for moving away from Nomad: too expensive and too complex. I ran Consul, Nomad and Vault on the same master instances (which is a risk and against best practice) without mTLS (just TLS) and it took a week to get it all running. None of that includes monitoring, backups, auto-scaling, etc.

I believe the work involved with going DIY on the orchestration front is not worth the benefits in the short to medium term. In the long term you'll likely outgrow your own simpler solution and at which point you'll likely have the capital to switch to a managed service anyway /shrug

It's all relative, after all.

[Axsuul]

There's no need to go truly production grade initially. I run some projects in production on single node deployments (with Nomad, Vault, and Consul all on one node, i.e. bootstrap_expect = 1) and it's solid. The hardware specs required are no different than if you were to just use systemd to run everything.

The beauty with HashiStack is you don't need to go fully monty. There's also no need to use ACLs which I agree adds a lot of unnecessary complexity if you're only at the MVP phase or doing things solo/in a small team. And you can run everything on a Digital Ocean $10/mo droplet -- it's very lightweight. I even have one HashiStack deployed on a Raspberry Pi running Home Assistant and other ancillary wares and it works great. But there's always the path to building a proper cluster if you need the redundancy.

I'm very bullish on HashiCorp. They can do a much better job though of communicating that not every deployment needs to be enterprise-grade and offer better ways of bootstrapping the type of cluster that fits your needs. I think a lot of that messaging gets lost due to their ongoing war with Kubernetes and what large companies are looking for.

[movedx]

I actually totally agree with your approach, but not the technology stack choice.

I think if you're at the MVP/solo-team stage then a single server that you push the app too with Ansible/Puppet Bolt, run via systemd, and then monitor, is even simpler, easier and "cheaper" (in time, which is money) than having to learn and use the HashiStack.

My app compiles into a single binary thanks to Go's binary packing feature. Does deploying a single binary reeeeaaallly need the entire HashiStack? Seriously? Do I need TWO operating systems (the host's and the container's) to run a binary?

Honestly I just think people over think this stuff and are easily swayed by cool features.

[MonaroVXR]

But how do you provision the baremetal?