[Tabular-Iceberg]

This is the rationalisation I get every time when I tell companies that their data model is a mess. Never mind that neither I nor the parent said anything about doing it up front.

Of course they have to iterate, the problem is that there is no deliberate effort anywhere, it’s just piling more crap on top of old crap and deluding themselves that they are some kind of lean, agile visionaries because of it.

[tomnipotent]

No one gets it right, and it's just grandstanding to pretend that "deliberate effort" is the distinguishing difference between good or bad data models. Unless you're dealing with highly specialized technical scenarios, most software is written to solve ambiguous and nebulous business problems that even the business doesn't necessarily understand.

[Tabular-Iceberg]

> most software is written to solve ambiguous and nebulous business problems that even the business doesn't necessarily understand

Obviously if you're not deliberate about what problem you're trying to solve in the first place, no amount of deliberate effort will produce a good data model. Designing business processes must be done with the same deliberate effort, and also need to be constantly refactored.

[tomnipotent]

Have you worked at a start-up? Or in a field new to you, or new altogether? No amount of well-intended deliberate design will make up for domain expertise. Ever. The best you can do is mitigate as much future refactoring as possible, but eve than that's not always the best use of time.

> Designing business processes must be done with the same deliberate effort

That's not always realistic. Businesses processes solve existing problems, sometimes tackle new problems, but just like data models hindsight is 20/20. And just like data models, business processes are constantly evolving and never done.

[Tabular-Iceberg]

> Have you worked at a start-up? Or in a field new to you, or new altogether?

Yes, to both questions. And I still stand by everything I said.

In fact both cases made me form this exact opinion. Being all hand-wavy and haphazard about our business processes is precisely and without a doubt what killed it. The second example was from a big regulatory compliance thing that had taken the particular industry by surprise, again because nobody in that industry can be bothered to understand their own business processes, but cargo cult everything. Managers and domain experts would throw PowerPoint mockups over the fence and tell the devs to "just make it clickable and put it on the web site". I had to constantly fight them to get them to take their own jobs seriously. I nearly burnt out and had to quit shortly after, but we actually released on time and though not perfect, came out miles ahead of the competition.

> And just like data models, business processes are constantly evolving and never done.

That makes deliberate design of business process more important, not less. A culture of hand-waviness is precisely why businesses are still caught by surprise by things like regulations that they are given years in advance to implement. If you don't know where you are, you don't know what you're supposed to pivot to.

[Ericson2314]

Yes absolutely. From day 0 everyone should already know how to do data modeling and perform migrations. If you don't and call yourself and engineer, you have no business starting a company.

[laichzeit0]

I think this is largely a consequence of microservices. What is the "data model" here? You're thinking database, to a microservice that's a repository implementation detail.

[Tabular-Iceberg]

I don't think it's a consequence of microservices, I think the problem is as old as the programmable computer. I'm the old paper and even clay tablet processes had to be constantly refactored too.

And a common argument I hear from microservice fanboys is that their old monoliths were suffering from high coupling and low cohesion and that microservices help with that. But I don't see why they couldn't have just refactored their monolith to address the problems they were having.

[Ericson2314]

I hope this is sarcasm poking fun at why microservices are terrible and small companies should stop LARPing FANG and avoid them?

[laichzeit0]

I’m not saying micro services are terrible at all. But I think the low coupling leads to a lot of “problems” when you try and re-aggregate data for analytics purposes when fundamentally micro services are meant to enable teams to work on them without worrying about how some other micro service is storing its data. It’s a trade off you make.

[Tabular-Iceberg]

> fundamentally micro services are meant to enable teams to work on them without worrying about how some other micro service is storing its data

What exactly is it that microservices bring to the table to achieve that, that doesn't already exist out of the box in every single language in common business use?

[laichzeit0]

It lowers coupling between teams. If my team is responsible for microservice A then I have no dependency on the team working on microservice B. Microservice A writes to its own database. There are no migration dependencies, etc. It's a distributed system. You have to accept that one microservice could be using mongodb, another postgresql, another mssql. You're typically communicating through APIs, one microservice is not reading directly from the database of another microservice. Materializing all this crap when you want to do analytics is where the problem comes in as now you are forced to construct some unified data model.

[Tabular-Iceberg]

> It lowers coupling between teams.

Yes, but you don't need microservices to do that. You can just lower the coupling between the teams working on your monolith using the means of abstraction of the language is written in.

> If my team is responsible for microservice A then I have no dependency on the team working on microservice B.

How is that different from the team responsible for Java package A having no dependency on the team working on Java package B? Or whatever language you happen to use.

> Microservice A writes to its own database. There are no migration dependencies, etc. It's a distributed system. You have to accept that one microservice could be using mongodb, another postgresql, another mssql.

Nothing is stopping you from connecting to multiple databases from the same monolith either.

>You're typically communicating through APIs, one microservice is not reading directly from the database of another microservice.

You can (and should) build well defined APIs between modules in your monolith too. And it's faster than having to keep serializing and deserializing everything all the time.

>Materializing all this crap when you want to do analytics is where the problem comes in as now you are forced to construct some unified data model.

Why do you think you need to unify your data model in a monolith? If you think it's the wrong strategy for your problem all you have to do is not do it.