[franga2000]

And I don't understand the obsession with server-based databases for single apps. Especially in containerised setups, every "app" gets its own database anyways, and if the app is further broken down into services, they usually communicate between each other and not with a shared database. So in those cases, what do you gain by pulling the database out of the "process" and onto the other end of a socket? In most cases, absolutely nothing. So why bother?

Don't get me wrong, I've worked with plenty of server-based databases, including proper dedicated database servers. It's great tech and often the best tool for the job. But not always and I'd argue not in the majority of uses.

[stingraycharles]

“Especially in containerised setups, every "app" gets its own database anyways, and if the app is further broken down into services, they usually communicate between each other and not with a shared database. “

You seem to be talking about a vastly different use case.

Containerized apps having their own database? What? Aren’t these types of containers stateless? I always very much try to keep state out of app containers.

What kind of data storage are we talking about?

[franga2000]

If an app needs a database, it gets a database server container, instead of getting a user and database on a shared database server as things used to be done. Every little django app has its own postgres container. Every wordpress site gets its own mysql container. That is the modern way.

Those database containers get a PVC/volume/mount for their data dirs. The only thing ever connecting to them is their "owner" application container. So at that point, why not drop the postgres container and PVC mount a sqlite directory in the app container? The result is the same.

[jappgar]

And when you need to scale to thousands of instances of your microservice?

[stingraycharles]

Yeah this is the part I don’t get. It seems like people are talking about 1 distinct app = 1 container and this is the new normal? We’re back to managing cows instead of cattle again?

[jappgar]

I just think a lot of people here haven't ever worked on large scale systems. They don't know what the don't know.

[andersmurphy]

I think a lot businesses build large distributed systems prematurely. They don't know what they don't know.

[andersmurphy]

What on earth needs thousands of instances? Are you building a CDN? Most apps can be a single server or sharded by business/region.

Honestly, this whole leys run loads of nodes seems to have sprung up from languages that are slow oe don't have decent concurrency.

[graerg]

That's the whole thesis; YAGNI.

[mxey]

Yes if you run a database server like an embedded application database, then it won’t be very different from an embedded application database.

[cortesoft]

How do you do server maintenance or handle hardware failure if your database is SQLite? You are going to have to take downtime, even in the best scenario.

[franga2000]

1. Proxmox live migration or HA, Ceph storage

2. K8S DaemonSet, PVC backed by probably Ceph

3. Just..don't care? Do maintenance outside of working hours, fix issues quickly and explain things nicely to your customers. Not everything is google-scale. Most people can deal with some downtime.

And it's not like you won't have downtime in let's say a postgres-backed app. But now you have two "servers" to deal with.

[cortesoft]

Those first two options add more complexity than just using an external database.

I guess I am just not used to downtime being acceptable. Spent most of my career working for a CDN, any sort of downtime was simply unacceptable. I can't stop myself from that sort of thinking now.

[franga2000]

It's complexity you already have. You need some sort of HA for your app server and some sort of resilient storage for your database server. Using sqlite just means the storage is used by the app server directly, nothing more.

[HDThoreaun]

Every container gets its own database?

[franga2000]

Yes? Well, every "app", as I quite explicity wrote. Look up the docker compose file or helm chart for basically any app. I'm running dozens of apps, each with their own postgres, redis and nginx containers alongside the main application server. That's what the stack is designed for.

[mxey]

The Compose file is written like that so you can quickly try the app without setting up extra dependencies. Usually not for production use.

Especially since in production you might want to scale the parts separately. I like to have a Postgres cluster to connect where backup is already handled, and the app then doesn’t have any persistent data, doesn’t need any network volume mounts.