[MisterKent]

Lot of kubernetes hate here, which is surprising. I run a little 3 node cluster and besides the hardware issues I had (long story), it has been rock solid and dead easy to setup.

Talos + longhorn + fluxcd (optional), is super nice. And everything beyond that is additive and just works within the ecosystem.

If anything, it helped keep my stuff alive during all the hardware issues a lot longer.

I think like 5-6 years ago, kubernetes on baremetal was pretty painful. People should really give it another try, an LLM can probably set it up for you and fire off the docker compose to manifests in one shot. Or just follow the docs yourself, maybe a dozen commands to get a cluster running?

All the enterprisey stuff makes it feel a lot more complex than it really is.

[dspillett]

> an LLM can probably

That would not match their reason for preferring to move off k8:

> I built a system that I didn't actually know how to maintain without the time or energy necessary to dig myself out of trouble.

If you want a system that you can understand all at once in your own head then making it with your own head is a better way to go IMO, for a small system at least.

[dewey]

When you are only used to the Kubernetes at work I can understand how people dislike it. If you set it up yourself and start with minimal feature and not lots of annotations the config files become very simple and not more complicated than a docker compose file.

It quickly realized that after just using the managed Kubernetes from Digital Ocean and deploying a side project there.

[PunchyHamster]

Oh running stuff on k8s is great, but running k8s itself is just adding a lot of code to your stack and might be not so trivial to debug when something goes wrong.

Then again tools to deploy it went a long way

[dewey]

Agreed, personally I'd only do it through a hosted provider or maybe consider https://k3s.io for a bit simpler setup. I'd also only do it if Kubernetes is something I'm already familiar with.

[wannabe44]

> All the enterprisey stuff makes it feel a lot more complex than it really is.

Kubernetes is not PaaS. It is a low level container orchestrator. But people start by thinking of it as a ready-made PaaS and then keep on trying to make it closer to that vision by continually slapping random plugins on it.

In corporate environments, incompetence is the norm. Every kubernetes deployment in such places ends up relying on rando docker images, helm charts and "operators" written by script kiddies and webshits which break every night and are hardly better than coding the same functionality yourself.

You need a strong instinct to say "No" and taste to keep the system complexity under reasonable limits. Sometimes this involves actually architecting your "platform" code around the barebones orchestration provided by kubernetes instead of slapping on latest webshit shown on CNCF dot org. That sort of judgement and taste is impossible to find in corporate environments in the age of resume driven development. That is how most people get Kubernetes scars.

[MisterKent]

Right.

If you are used to docker compose and don't care about redundancy or authoring your own stuff. Just setup storage and then use helm install. It feels like an app store at that level.

Grow from there as needs arise.

[vachina]

> Talos + longhorn + fluxcd

Are you running a startup at home.

[lachiflippi]

That's, in my opinion, a pretty vanilla starter stack. Could probably argue that you don't need replicated storage, and that ArgoCD is nicer to look at, but Talos in particular simplifies a lot of the deployment because there's not much of an OS to configure or keep updated.

[zrail]

Sure! That's the setup I was running before all of this. My problems are likely entirely my own making, tbh, as described in the article.

IME K8s in general and longhorn especially behaved poorly in the face of heterogeneous hardware where the largest node is flaky and has more cores and memory than the rest of the nodes combined.

My point is that, for me, in my setup, with my own idiosyncratic constrains, trying to keep up with Talos + longhorn + fluxcd expended way too many spoons in any given month.

[felooboolooomba]

> Lot of kubernetes hate here, which is surprising

It's easier when you think of it using cats. A person wants a cat. They want a big one to deal with their rodent problem. They've heard that tigers are really big cats and extremely efficient in dealing with unwanted biomass around them.

Said person gets a tiger. They get an impromptu expedition to hell and barely make it back. Person hates cats, buys a poodle and advocates for mouse traps.

[maxrecursion]

That's a very accurate and elegant analogy.

Kubernetes was designed for large organizations with massive applications that are usually micro services. It allowed the applications to be load balanced and easily updated with no downtime. Also, these organizations paid top dollar to highly technical and competent employees.

Now, all organizations feel like they need kubernetes because it's the next logical step in the never ending IT progression. So, they force all their apps to be containerized and run on an multi-tenant cluster.

However, these apps worked just fine on a server, and now the devs have to refactor them and get them to work on kubernetes. All the System Admins and Engineers also need to learn kubernetes. But most of these people don't want to constantly learn and use kubernetes, so no one ever really knows how it all works.

This inevitably led to companies whose sole purpose is to help places get off of kubernetes because it is too hard to maintain. I've seen it play out at several organizations.

[pjmlp]

I got introduced to UNIX with Xenix, have used plenty of flavours including containers in HP-UX and Solaris before they became a thing in Linux, I have zero need to use Kubernetes at home.

In fact, I also have zero needs for it at work directly, as I have become an advocate for serverless and managed runtimes, unless there is really a business need to control the whole infrastructure, including the Kubernetes cluster directly.

[PunchyHamster]

It's all nice till something breaks and you need to debug something, and when it does the less layers you have the easier it becomes

> All the enterprisey stuff makes it feel a lot more complex than it really is.

It is insanely complex under the hood. We just got to level when for most cases the tools available work just fine.

[gbalduzzi]

Did you already know kubernetes, or did you have to learn it for this setup?

I believe this is the key difference, because it doesn't make sense to learn such a complicated tool for a simple use case.

If you already know it sure, it may make sense. But you are removing a huge initial effort required otherwise.

[MisterKent]

I learned it for this setup. I do zero devops at work, there's a separate team that handles that (and even then, they're not on kubernetes).

I learned it because it seemed like the right way to do things and fun. Cut myself a few times in the earliest days having no clue, but no regrets.