Hacker News new | ask | show | jobs
by darken 935 days ago
I was in a similar boat and finally got around to trying out Proxmox in the last year or so. I just wanted share my own experience here, since it's a bit different than the "standard" uses I've seen.

I had been running a k3s cluster (k8s flavor) on some Raspberry Pis, but decided I needed some non-ARM nodes, and "invested" in a few low power "1L" AMD64 PCs (6-8 core + hyper-threading). I was initially going to just install Ubuntu and base my setup off my existing Ansible automation to make things less inefficient. But I figured I'd play around with Proxmox first and see if there was any benefit to using that as a base layer since I'd heard a lot about it.

I'm so glad I did. I ended up learning quite a bit in the process. Some quick highlights about using Proxmox for VMs in general:

* Proxmox supports creating a "cluster", so you can login though one machine to administer them all. You can conveniently "move" VMs between machines pretty seamlessly.

* If you install the para-virtualization drivers for e.g. Windows or Ubuntu VMs, you can do pretty fast remote KVM. E.g. I could run Youtube on a Windows VM in my basement over "Spice" and it almost looks like it's running locally. (Not that it's a use case I care about, mostly just shows the fact it's performant.)

In terms of actually getting around to deploying k3s on top of the infra:

* I ended up learning HPC-Packer, and HPC-Terraform, which integrated nicely with my existing Ansible experience.

* Packer turns an Ubuntu ISO + my "base" Ansible setup playbooks into a pre-baked machine template directly in Proxmox. (My local machine's Packer binary just orchestrates the process.)

* Terraform deploys the machine template into the Proxmox cluster. Basically a config file of machine names + IPs + mac adresses, and a few other params and initial setup.

* Ansible then installs any final dependencies (anything not in the base template), setups up the first k3s master, grabs the join token, and adds in 2 more master nodes for a proper `etcd` backend.

* Ansible then installs my base Kubernetes services (cert-manager, Rancher, Longhron storage, etc) via running helm commands on one of the nodes.

* This is where I'm at now; the next step is for me to deploy my existing Flux.cd-automated "Gitops" apps (built for ARM64+AMD64 via Gitlab runner, also in Proxmox). These _had_ been running on my now-quite-crusty-seeming Pi cluster.

I can run a single command to delete all the VMs, and rebuild + setup everything (full HA cluster + apps deployed and running) from scratch in ~6 minutes without any manual input required from me, just a few secrets/params in a config file.

This has made exploring the horizon of possibilities _so much easier_ without getting locked in; I can try to weird Longhorn storage configs, or try out k8s monitoring stacks without worrying about needing to "back out" my changes if I picked bad settings. (Just blow it up and try again!) I can change how VLANs are configured in early steps, or try adding a library to the base Ubuntu install cluster-wide super easily, etc.

I am primarily a software-engineer, so it has been really nice to delve into the operational side of things, and get a proper reproducible setup. It really has transformed how I think about the cluster in that it's no longer a "thing to carefully maintain", but instead a great sandbox to explore AND deploy my own k8s applications on top of without playing cloud bills.

My Proxmox journey in the past few months definitely turned into more than a rabbit hole than I'd expected.

2 comments

This is a great answer, thank you! It really makes sense to build a box with Proxmox at home with this view. Nice!
Running it at home on a few 1L thin clients is pretty grand. Quiet, performant, and everything you learn scales to enterprise well.
Could I please ask you to share some of your Terraform files? I'm looking to build a similar setup on my Proxmox cluster, and it would really help to have a template to go from!