Hacker News new | ask | show | jobs
by devjab 625 days ago
I disagree rather strongly with this advice. Mostly because I’ve spent almost a decade earning rather lucrative money on cleaning up after companies and organisations which did it. Part of what you say is really good advice, if you’re not Facebook then don’t build your infrastructure as though you were. I think it’s always a good idea to remind yourself that StackOverflow ran on a few IIS servers for a long while doing exactly what you’re recommending that people do. (Well almost anyway).

Using a single database always ends up being a mess. Ok, I shouldn’t say always because it’s technically possible for it not to happen. I’ve just never seen the OOP people not utterly fuck up the complexity in their models. It gets even worse when they’ve decided to use stores procedures or some magical ORM which not everyone understood the underlying workings of. I think you should definitely separate your data as much as possible. Even small scale companies will quickly struggle scaling their DBs if they don’t, and it’ll quickly become absolutely horrible if you have to remove parts of your business. Maybe they are unseeded, maybe they get sold off, whatever it is. With that said, however, I think you’re completely correct about not doing distributed transactions. I think that both you and the author are completely right that if you’re doing this, then you’re building complexity you should be building until you’re Facebook (or maybe when you’re almost Facebook).

A good micro-service is one that can live in total isolation. It’ll full-fill the need of a specific business domain, and it should contain all the data for this. If that leaves you with a monolith and a single shared database, then that is perfectly fine. If you can split it up. Say you have solar plants which are owned by companies but as far as the business goes a solar plant and a company can operate completely independently, then you should absolutely build them as two services. If you don’t, then you’re going start building your mess once you need to add wind plants or something different. Do note that I said that this depends on the business needs. If something like individual banking accounts of company owners is relevant to the greenfield workers and asset managers, then you probably can’t split up solar plants and companies. Keeping things separate like this will also help you immensely as you add on business intelligence and analytics.

If you keep everything in a single “model store”, then you’re eventually going end up with “oh, only John knows what that data does” while needing to pay someone like me a ridiculous amount of money to help your IT department get to a point where they are no longer hindering your company growth. Again, I’m sure this doesn’t have to be the case and I probably should just advise people to do exactly as you say. In my experience it’s an imperfect world and unless you keep things as simple as possible with as few abstractions as possible then you’re going to end up with a mess.

2 comments

I wanted to respond to you, because you had some excellent points.

> Mostly because I’ve spent almost a decade earning rather lucrative money on cleaning up after companies and organisations which did it.

For many companies, this is actually a pretty successful outcome. They built an app, they earned a pile of money, they kept adding customers, and now they have a mess. But they can afford to pay you to fix their mess!

My rule of thumb of "design for 10x scale" is intended to be used iteratively. When something is slow and miserable, take the current demand, multiply it by 10, and design something that can handle it. Sometimes, yeah, this means you need to split stuff up or use a non-SQL database. But at least it's a real need at that point. And there's no substitute for engineering knowledge and good taste.

But as other people have pointed out, people who can't use an RDBMS correctly are going to have a bad time implementing distributed transactions across microservices.

So I'm going to stick with my advice to start with a single database, and to only pull things out when there's a clear need to scale something.

Well, I guess the side of my argument which is missing by my anecdotal experiences is that monoliths is what I work on because it was the trend. It’s probably easier to fix the complicated mess of a monolith than a complicated mess of micro-services done wrong.
All that you say is true, and people who do that are THE LESS capable of becoming better at the MUCH harder challenges of microservices.

I work in the ERP space and interact with dozens and I see the horrors that some only know as fair tales.

Without exception, staying in an RDBMS is the best option of all. I have seen the cosmical horrors of what people that struggle with rdbms do when moved to nosql and such, and is always much worse than before.

And all that you say, that is true, hide the real (practical) solution: Learn how to use the RDBMS, use SQL, remove complexity, and maybe put the thing in a bigger box.

All that is symptoms that are barely related to the use of a single database.

What I dislike about single databases is that it’s too easy for people to build unnecessary relationships. You obviously don’t have to do it and there are a lot of great tools to separate data. That’s not what people are going to do on a Thursday afternoon after a day of horrible meetings though. They’re going to take shortcuts and mess things up if it’s easy to do so. Having multiple databases, and they can all be SQL (should if that’s what your developers know), in isolation is to protect you from yourself, not so much because it’s a great idea technically.
But that is the same if you have many databases. Only that the problem spread!

Maybe is because we are in different niches?. In mine, I have never seen microservices having ANY improvement over the norm, and most certainly are far more negatives.

However, what is more, the norm is making a 2/3-tier from a monolithic, and that could be better.

P.D: In the ERP/business space you can have many, whole apps, with ETL in the middle orchestrating. That may improve things because the quality of each app varies, but what is terrible is to split apps into micro services. That is itself a bridge too far.