[0x457]

I don't understand some home labs. You see a beefy, but old rack server that has next to none single-thread performance (relatively). Usually servery underutilized un running Proxmox (issa lab after all) and RPi doing something for some reason?

Those setups always pure "home-lab" because it's too small or macgyvered together for anything, but the smallest businesses...where it will be an overkill.

Sometimes it's people running 2-3 node k8s cluster to run a few static workloads. You're not going to learn much about k8s from that, but you will waste CPU cycles on running the infra.

[NathanKP]

Running stuff on an underpowered Raspberry Pi is a good way to sniff test whether an infrastructure or software setup is sane. Powerful computers can hide horrible decisions for a long time, while less powerful devices make it immediately obvious if you need to switch a more efficient configuration.

I think this is one of the main reasons why Raspberry Pi has such a strong representation in homelabs, including my own.

[0x457]

> I think this is one of the main reasons why Raspberry Pi has such a strong representation in homelabs, including my own.

I'd bet you're minority. People use Pi because it lets them assemble a cluster for under 200 bucks.

[_dain_]

the point of the hobby is to have something to tinker with. if it were too easy it wouldn't be any fun.

[0x457]

I know, that's why I have my own "lab". I just don't get why most of other labs are so cookie-cutter proxmox + home assistant + (unifi controller) + pihole and there is always RPi somewhere next to a chunky server.

[perdomon]

Because people have fun creating those sorts of setups. There are many tutorials for those setups, the hardware can be found relatively affordably, and it isn't something someone needs to have experience in enterprise networking to build. The purpose of the hobby is to have fun. If you aren't having fun, or if other people's fun is ruining your day, consider finding a new hobby. I suggest fly fishing. You could probably use more vitamin D.

[0x457]

> You could probably use more vitamin D.

Nah, I get enough of that from being by the pool, managing my home lab.

[unethical_ban]

Someone who successfully configures and tests a k8s cluster at home knows more about it than I do.

[0x457]

You overestimate the complexity. Even if you don't use k8s installer/distribution and do it from scratch - you are not going to learn much about operating/using k8s.

[udev4096]

I see people like you on /r/homelab all the time. "What's the point of this?" "k8s is not for you". Shut the fuck up man. It's called homelab for a reason. People can design it however they want, even if it's underutilized. And, you can learn a lot by running k8s cluster at home.

Start with k3s, configure ingress/services/deployment, install ingress controller for cluster wide routing, install service meshes (istio, cilium), write a controller, mess around with gateway API. The possibilities are endless. Stop bitching and maybe have a real argument instead of shitting on homelabbers

[0x457]

> you can learn a lot by running k8s cluster at home.

That is my argument, you're not learning much if your cluster 2-3 machines and a few RPis.

> Stop bitching and maybe have a real argument instead of shitting on homelabbers

Why so rude? I have a homelab myself, it's just for running things and not LARPing as sysadmin.

[udev4096]

Only a real sysadmin, which you clearly aren't, would know if someone is "LARPing as a sysadmin". So stop trying to pretend like a know-all smartass and GTFO. As the other comment said, get some vitamin D instead of spreading totally misleading statements on things you clearly don't know anything about

[otter-in-a-suit]

> none single-thread performance (relatively)

I find horizontal scaling with many smaller cores and lots of memory more impactful for virtualization workloads than heavy single core performance (which, fwiw, is pretty decent on these Xeon Golds).

The biggest bottleneck is I/O performance, since I rely on SAS drives (since running full VMs has a lot of disk overhead), rather than SSDs, but I cannot justify the expense to upgrade to SSDs, not to mention NVME.

> Those setups always pure "home-lab" because it's too small or macgyvered together for anything, but the smallest businesses...where it will be an overkill.

That is a core part of the hobby. You do some things very enterprise-y and over-engineered (such as redundant PSUs and UPSs), while simultaneously using old hard drives that rely on your SMART monitor and pure chance to work (to pick 2 random examples).

I also re-use old hardware that piles up around the house constantly, such as the Pi. I commented elsewhere that I just slapped an old gaming PC into a 4U case since I want to play/tinker with/learn from GPU passthrough. I would not do this for a business, but I'm happy to spend $200 for a case and rails and stomach an additional ~60W idle power draw to do such. I don't even know what exactly I'll be running on it yet. But I _do_ know that I know embarrassingly little about GPUs, X11, VNC, ... actually work and that I have an unused GTX 1080.

Some of this is simply a build-vs-buy thing (where I get actual usage out of it and have something subjectively better than an off the shelf product), others is pure tinkering. Hacking, if you will. I know a website that usually likes stuff like that.

> You're not going to learn much about k8s from that

It's possible you and I learn things very differently then (and I mean this a lot less snarky than it sounds). I built Raft from scratch in Scala 3 and that told me a lot about Raft and Scala 3, despite being utterly pointless as a product (it's on my website if you care to read it). I have the same experience with everything home lab / Linux / networking - I always learn something new. And I work for a networking company...

[0x457]

> It's possible you and I learn things very differently then (and I mean this a lot less snarky than it sounds). I built Raft from scratch in Scala 3 and that told me a lot about Raft and Scala 3, despite being utterly pointless as a product (it's on my website if you care to read it). I have the same experience with everything home lab / Linux / networking - I always learn something new. And I work for a networking company...

Building k8s from scratch, you're going to learn how to build k8s from scratch. Not how to operate and/or use k8s. Maybe you will learn some configuration management tool along the way unless your plan is to just copy-paste commands from some website into terminal.

> find horizontal scaling with many smaller cores and lots of memory more impactful for virtualization workloads than heavy single core performance (which, fwiw, is pretty decent on these Xeon Golds).

Yeah, if you run a VM for every thing that should be a systemd service, it scales well that way.

[christphrdunder]

> should be a systemd service

"should be" according to your goals. "should not be" according to mine:

1. run untrusted code in a reasonably secure way. i don't care how many github stars it has, i'm not rawdogging it. nor is my threat model mossad, so it doesn't have to be perfect. but systemd's default security posture is weak, hardening it is highly manual (vs. "run a VM"), and must be done per service to do properly (allowlist syscalls, caps, directory accesses, etc.).

2. minimize cost. it's orders of magnitude less costly for most people with the skills to run a homelab to:

   a) spend 50% more for compute & storage than a single hour of handwriting & tuning reasonably secure systemd services, wiring up dependencies, etc.

   b) build a backup and migration strategy once and reuse it for everything. this is technically possible with systemd, too, of course, but way more costly to setup.

   c) one single universal solution. practically everything will run in a VM. this is not true for systemd, esp. isolated systemd services.

if you want to optimize for "learn how to configure systemd", "learn how to hyperoptimize cpu usage", or whatever it may be then great. if other people aren't, they're not necessarily wrong, they may be choosing different tradeoffs. understanding this is an essential step in maturing as an engineer and human being. i truly mean this as encouragement - not rebuke. Otherwise i wouldn't have paid the relatively high cost in time to write it afterall :)

[udev4096]

OP is right (not surprising, given he's a platform engineer with decades of experience unlike you, a nobody). You are projecting your extremely flawed and naive approach of learning k8s. If all you could think of is "copy-paste configs" then sure, you are definitely NOT gonna learn anything. You're just trying to act smart with absolute hatred towards people running k8s outside of businesses

[0x457]

> (not surprising, given he's a platform engineer with decades of experience unlike you, a nobody)

Also, an engineer with experience? I'm calling out over-engineering for the sake of over-engineering.