Hacker News new | ask | show | jobs
by darken 1195 days ago
This went a different direction than I'd expected. That is, configuring your cluster resources rather than the host machines themselves.

I managed to grab 5 or so Raspberry Pi 4s before the availability issues. I've been (slowly) getting a home Kubernetes cluster running over the last year or two, and Ansible helps immensely; I:

* Write Ubuntu 20.04 (or now 22.04) to MicroSD card.

* Configure the initial user + SSH-PK, hostname, and network via config added to the SD

* Boot up the new Pi

* Run Ansible playbook to get it ready for k3s (lightweight but "full" k8s); including bind-mounting all high use paths for containerd/logs/etc to a USB NVME-SSD, adding boot args to the Pi's GRUB, etc.

* Push all my bashrc/tmux/vim/etc configs for consistent environment if I ssh in.

* Use Ansible ad-hoc for misc changes or package updates.

In terms of the "deploying to Kubernetes", I've recently got FluxCD working and love it. I think "GitOps" is the way to go in terms of clear reproducible config.

I have GitLab running on a local Synology (NAS) which kicks off any container builds and hosts the Git and container registry. The FluxCD operator on the cluster polls my "cluster as code" repo and syncs state. Each project defines a Helm chart and Flux pulls the chart from each repo and applies any variables (as configured in the "cluster repo") or "kustomizations".

Flux let's you setup "dependency" relationships for components. So I have it add the sealed-secrets operator early in, and put PK-encrypted secrets in directly the Git repo for it to"unpack" in the cluster.

That being said, the author's Anisble-driven approach is a much quicker way get something on a cluster ASAP, versus setting up the initial Flux "pull from cluster" system.

I'm primarily a software engineer, so this was also largely an exercise in getting more comfortable with the operations side of things.

Just thought I'd share my experience since only so many people let me rant about my FluxCD/k3s/GitLab home dev environment.

2 comments

`kubectl apply -f myshit/` is a faster way of getting things on your cluster, ansible is just a bunch of wrapper on top of that.
And when you outgrow it you can move on to `kubectl apply -k`...
Until you need to update it, or apply it to multiple environments, or any number of tasks where the concepts of configuration parameters and dependencies become important.

Edit: I'm not saying Ansible is an appropriate substitute (it absolutely isn't), rather that `kubectl -f` is not a scalable solution.

All I'm saying is ansible is essentially that and no it isn't at all scalable.
Some of the other controllers in the Flux family are also nice. There's also the Terraform and Helm controllers and they all work in a consistent way and produce consistent metrics about what they're doing