[asim]

> Microservices only pay off when you have real scaling bottlenecks, large teams, or independently evolving domains. Before that? You’re paying the price without getting the benefit: duplicated infra, fragile local setups, and slow iteration. For example, Segment eventually reversed their microservice split for this exact reason — too much cost, not enough value.

Basically this. Microservices are a design pattern for organisations as opposed to technology. Sounds wrong but the technology change should follow the organisational breakout into multiple teams delivering separate products or features. And this isn't a first step. You'll have a monolith, it might break out into frontend, backend and a separate service for async background jobs e.g pdf creation is often a background task because of how long it takes to produce. Anyway after that you might end up with more services and then you have this sprawl of things where you start to think about standardisation, architecture patterns, etc. Before that it's a death sentence and if your business survives I'd argue it didn't because of microservices but inspite of them. The dev time lost in the beginning, say sub 200 engineers is significant.

[candiddevmike]

Some resume driven developers will choose microservices for startups as a way to LARP a future megacorp job. Startup may fail, but they at least got some distributed system experience. It takes extremely savvy technical leadership to prevent this.

[devin]

In my experience, it seems the majority of folks know the pitfalls of microservices, and have since like... 2016? Maybe I'm just blessed to have been at places with good engineering, technical leadership, and places that took my advice seriously, but I feel like the majority of folks I've interacted with all have experienced some horror story with microservices that they don't want to repeat.

[Espressosaurus]

I feel like it's only in the last 5 years in the tech publicity sphere that I've seen pushback against microservices, only it feels like only the last year or two where I see it to the exclusion of influencers pushing microservices.

Things are different in the embedded space so I don't have personal experience with any of it.

[speed_spread]

Pushback was always there from the start. The first edition of O'Reilly's "Building microservices" recommended _against_ microservices, unless you absolutely tried scaling your monolith and team beforehand.

Any organization stuck in microservice hell fully deserves the punishment.

[jamesfinlayson]

> In my experience, it seems the majority of folks know the pitfalls of microservices, and have since like... 2016?

Majority maybe, but not everyone. I was at a place in 2018 where a guy turned up declaring that microservices would solve all of the company's performance issues (he'd heard that AWS S3 was made up of over 500 microservices so we must do the same, but he obviously had no idea about the depth of features in S3 so hit take was more is better).

So we got like 30 microservices in the space of year which gave the company 10x complexity, complicated and fragile local development and... dubious wins like faster email sending, but no improvements to true bottle-necks of the system.

[westurner]

Does [self-hosted, multi-tenant] serverless achieve similar separation of concerns in comparison to microservices?

Should the URLs contain a version; like /api/v1/ ?

FWIU OpenAPI API schema enable e.g. MCP service discovery, but not multi-API workflows or orchestrations.

(Edit: "The Arazzo Specification - A Tapestry for Deterministic API Workflows" by OpenAPI; src: https://github.com/OAI/Arazzo-Specification .. spec: https://spec.openapis.org/arazzo/latest.html (TIL by using this comment as a prompt))

[hnthrow90348765]

Hiring will need to change to stop resume-driven development (can't eliminate it completely though), because you're likely to only get monolith roles if you only work on monoliths. Only being able to speak about microservices puts you in the "talk the talk, not walk the walk" category.

It would also nice to have less fear-driven career advice like "your skills go out of date" which drives people to try adopting the latest things.

[Mountain_Skies]

Keyword driven and filtered application processes also heavily incentivize adding into projects whatever is being posted on jobs sites. If microservices are part of a company's standard template for developer postings, people who want to work at that company will find a way to get it on their resume.

[MDGeist]

I've also seen the top down version where senior leadership like a CIO/CTO wants to put a huge "modernization" project on their resume and they don't care if it is impossible to maintain or falls over after they move on.

[jamesfinlayson]

Oh yeah, I've come to realise that the people at the top are just as bad. I was at a place where "consultants decided" that the company's two backbone systems should be replaced with off-the-shelf solutions. Very obviously a resume-padding project from the CIO who was in their first large company job, but which would have almost certainly crippled the company if it went ahead.

Thankfully the business and finance people at the company decided it would be an expensive and stupid idea it was shut down.

[ang_cire]

"Cloud Migration"

[gpi]

"Digital transformation"

[alaithea]

And when it's your technical leadership leveraging buzzword-driven development to rise to the top, you're screwed.

[wpollock]

So true. It was in March that I saw on HN an advertisement for a vibe coder with 3 years experience. I believe the term "vibe coding" was invented a month before! Buzzword hiring is as bad as resume driven development.

[bityard]

It could also just be plain old overengineering. Like using Django and leaning on all of the magic contained within it just to implement a simple API that could instead be a very small Flask or FastAPI app.

[about3fitty]

Unforeseen scope creep is the reason to utilise Django over Flask, I feel.

Also, you can pick and choose what to use in Django similarly to Flask - it just has a higher initial learning curve.

Once you get to sufficient levels of complicated, leaning on established, documented, community supported design patterns and abstractions helps vs. sorting out your imports, making bespoke design choices, and doing a bunch of non-core value producing work.

[0cf8612b2e1e]

I will forever beat the Django over Flask drum. Nothing ever stays simple. Maybe today it is one single view function, but tomorrow someone is going to want to add authentication, emails, whatever. Django might have features you do not need, but it would be an unusual situation where Django is holding you back vs Flask.

[dimal]

I saw one startup with about fifty engineers, and dozens of services. They had all of the problems that the post describes. Getting anything done was nearly impossible until you were in the system for at least six months and knew how to work around all the issues.

Here’s the kicker: They only had a few hundred MAUs. Not hundreds of thousands. Hundreds of users. So all this complexity was for nothing. They burned through $50M in VC money then went under. It’s a shame because their core product was very innovative and well architected, but it didn’t matter.

[jghn]

> They only had a few hundred MAUs

Way too many companies believe they're really just temporarily embarrassed BigTech.

[danielscrubs]

Bad software dev. degrees that focus on fancy architecture that brings nothing to the table except overhead.

[betterThanTexas]

I don't think I learned basically anything about "fancy architecture" from my undergraduate courses except, ironically, reasoning about coupling and overhead.

[ang_cire]

I don't remember one solitary lecture on CI/CD, microservices, or even just deployment in general, in Uni. The closest that our comp. sci. classes ever came to touching on anything but the code itself was making us use SVN.

[danielscrubs]

The difference between a software dev degree and a comp. sci degree.

[ang_cire]

I've never heard of or seen a Software Development bachelors degree?

I've seen Information Systems programs, that are usually CE, after-hours tracks. Neither Harvard, Yale, nor MIT have a software dev one, just Comp. Sci. I'm calling BS (no pun intended) on "software dev degrees" as a thing distinct from CS in any widespread fashion.

https://www.harvard.edu/programs/

https://admissions.yale.edu/majors-and-academic-programs

https://facts.mit.edu/degrees-majors/

[DanielHB]

I have worked with a company with ~100k MAUs with ~4 teams, even then it often feels the system is over-microserviced (about two dozen services I think).

Definitely some stuff makes sense (especially since it has a lot of IoT stuff), but micro-service added was used mostly as a way to develop new stuff without having to deal with the legacy monolith. The core of the application could easily be a single service backed by one big RDBMS with a few ancillary services around it.

The legacy monolith is still there kicking and screaming, it didn't need "breaking up" it needed (and I assume still do) need a major refactoring.

[dakiol]

I’m not sure where’s the downside. The engineers got paid, they managed to put “founder” on their cvs, and enjoyed the ride. Now they are more prepared for their next adventure. The only ones who lost money were the investors, but nobody cares about them.

[OutOfHere]

Do you not care about the users either? They too will lose the service if the company shutters.

[singron]

> You'll have a monolith, it might break out into frontend, backend and a separate service for async background jobs

And when you break these out, you don't actually have to split your code at all. You can deploy your normal monolith with a flag telling it what role to play. The background worker can still run a webserver since it's useful for healthchecks and metrics and the loadbalancer will decide what "roles" get real traffic.

[elevatedastalt]

If you are building the same binary for all microservices you lose the dependency-reduction benefit microservices provide, since your build will still break because of some completely unrelated team's code.

[roguecoder]

If it is possible for that other team to merge a broken build, you are doing it wrong.

If you are concerned about someone else breaking your thing, good! You were going to eventually break it yourself. Write whatever testing gives you confidence that someone else's changes won't break your code, and, bonus, now you can make changes without breaking your code.

[motorest]

> If it is possible for that other team to merge a broken build, you are doing it wrong.

This assertion is unrealistic and fails to address the problem. The fact that builds can and do break is a very mundane fact of life. There are whole job classes dedicated to mitigate the problems caused by broken builds, and here you are accusing others of doing things wrong. You cannot hide this away by trying to shame software developers for doing things that software developers do.

> Write whatever testing gives you confidence that someone else's changes won't break your code, and, bonus, now you can make changes without breaking your code.

That observation is so naive that casts doubt on whether you have any professional experience developing software. There are a myriad of ways any commit can break something that goes well beyond whether it compiles or not. Why do you think that companies, including FANGs, still hire small armies of QAs to manually verify if things still work once deployed? Is everyone around you doing things wrong, and you're the only beacon of hope? Unreal.

[roguecoder]

I haven't seen a broken build in at least nine years, not since I left the company with a merge process built out of bash scripts that took three hours and required manual hand-holding.

I am genuinely curious what situations you are seeing where builds are making it through CI and then don't compile.

It isn't always worth investing in quality, but when it is it is entirely possible to write essentially bug-free software. I've gone seven months without a bug in production and the one we saw we had a signed letter from product saying "I am okay if this feature breaks, because I think writing the tests that can verify this integration was going to take too long."

FAANG companies aren't prioritizing writing software well: they are prioritizing managing 50,000 engineers. Which is a much harder problem, but the management solutions that work for that preclude the techniques that let us write bug-free software.

One of the great things about startups is that it is trivial to manage five engineers, so there is no reason we have to write software badly.

[motorest]

> (...) it is entirely possible to write essentially bug-free software (...)

You lost what little credibility you had left.

[JackSlateur]

You are absolutely right.

Of course, if people wrote bug-free code, then there would be no bug !

Bug-free code in the actual code, or bug-free code in the test code, this is the same story.

If you write stuff and never have any bug, then either:

  - you are lying
  - you do not write much
  - you only write really simple things
  - you are Jesus, came back from heaven to shine his light on us, poor souls

The more complicated, intricate stuff you have, the more bugs you'll get (and only time will allow you to fix that).

Tests are great do define how you think it should work, and to ensure it keeps doing that way. Take the time to think about the third point on the bullet list above.

[tuckerman]

Even if it builds successfully, I've never worked anywhere where automated tests prevented 100% of problems and I doubt I ever will. For most systems of sufficient complexity you are testing in prod, even if you did a lot of testing before prod as well.

[roguecoder]

That's even more true for microservices, though, since I have yet to see a microservice architecture that automatically runs end to end tests before deploying.

The post I was replying to said "your build will still break": that's what I was taking issue with. In this day and age there is no reason our trunk build should ever be broken.

[mjr00]

> I have yet to see a microservice architecture that automatically runs end to end tests before deploying.

One of the big tenets of independent services is that your APIs are contracts that don't change behaviour. As long as each individual service doesn't introduce breaking changes, the system as a whole should work as expected. If it doesn't this is indicative of either 1) a specific service lacking test coverage, or 2) doing something wrong i.e. directly reading from a microservices' database without going through an API.

[tuckerman]

Yes, I suspect some of the back and forth is the fuzziness of the term "broken build", whether that means the code literally doesn't compile or it does but the code does the wrong thing.

I agree that you can prevent merges that cause compilation errors in nearly all cases!

[jimbokun]

What about when it’s you breaking your own thing?

A very large code base full of loosely related functionality makes it more and more likely a change in one part will break another part in unexpected ways.

[DanielHB]

You have a point, but I wouldn't say this is a big deal unless there is a mammoth dependency somewhere that slow down things to a crawl. Then maybe that one part of the codebase can be broken into its own separate service.

But even then there are ways around this kind of problem with dynamic linking pre-built binaries and caching, but it is extra complexity that could be worse than managing multiple services. Docker cache can usually handle this pretty well though.

[jounker]

You’ll still get some isolation since not all pathways share the same code. It’s not all or nothing.

[tstrimple]

I put my team through this as an inexperienced lead about 15 years ago. We were a team of less than a dozen who had a nice single solution file that you could build and run the entire stack from. At the end we were looking at roughly a dozen services all which required orchestration to get them running and working together. First hand lessons in YAGNI and "do the simplest thing that works" which have stuck with me the rest of my career.

[jayd16]

it's weird that the your quote and your own explanation offer technical reasons for separate services but then you say it's not a technical pattern.

You'll need services. They're hard. If something is hard but it needs to be done, you should get good at it.

Like every fad, there a backlash from people seeing the fad fall apart when used poorly.

Services are a good pattern with trade offs. Weigh the trade offs, just don't do things to do them.

[jimbokun]

I thought the linked article about how Khan Academy eventually migrated to multiple services was a good example of when introducing micro services is a good idea:

https://blog.khanacademy.org/go-services-one-goliath-project...

They had already scaled the mono service about as far as it could go and had a good sense of what the service boundaries should be based on experience.

[fallingknife]

There are plenty of tech reasons for microservices. e.g. scaling high traffic services separately and separating low priority functionality from critical paths. I would agree that this is usually not a smart thing to do in a small org, but I have seen times where splitting out a high load path into a microservice has been very much worth it at a startup.

[bluefirebrand]

> scaling high traffic services separately

This is a great optimization once you have high traffic services

Building this way before you have any traffic at all is a great way to build the wrong abstractions because your assumptions about where your load will be might be wrong

[gopher_space]

Microservices are a technical solution to regional availability and pairing problems, and they start with a spreadsheet telling you when to make them based on requirements vs. cost. They're slow, expensive threads you should have a really good reason to use.

> Building this way before you have any traffic at all is a great way to build the wrong abstractions

These services only make sense to think about within specific traffic contexts. It'd be impossible to build the right abstraction.

[motorest]

> Microservices only pay off when you have (...) independently evolving domains.

I don't see any major epiphany in this. In fact, it reads like a tautology. The very definition of microservice is that it's an independently evolving domain. That's a basic requirement.

[asdf6969]

> Microservices are a design pattern for organisations as opposed to technology

Very true in my experience. The main benefit is letting small groups of people work independently without stepping on each other’s toes. Although I’ve worked on a project where multiple teams owned micro services that were supposed to be standardized with each other, and it just lead to endless meetings and requirements churn since nobody was willing to work on the other teams service but everyone had an opinion on what the cross-team standard should be. Learning the diplomatic way to say “mind your own business” was more important than any technical skills for getting code merged.

[PaulHoule]

I've tended to use microservices in limited cases where the system had to serve a few requests that had radically different performance requirements, particularly memory utilization. I had a PHP server for instance that served exactly one URL for which PHP was not a good fit and a specialized server in another language for that one URL gave like 1000x better performance and money savings in terms of not needing a much bigger PHP server.

Using Spring or Guava in the Java world it is frequent that people write "services" that are simply objects that implement some interface which are injected by the framework. In a case like that you can imagine a service could have either an in-process implementation or an out-of-process implementation (e.g. via a web endpoint or some RPC.) Frameworks like that normally are thinking at the level of "let's initialize one application in one address space at a time" but it would be nice to see something oriented towards managing applications that live in various address spaces.

Trouble is that some people get this squee when they hear they can use JDK 9 for this project and JDK 10 for another project and JDK 11 for another project and they'd rather die than eschew the badly broken Python 3.5 for something better. If you standardized absolutely everything I think you could be highly productive with microservices because you wouldn't have to face gear switching or deal with teams who just don't know that XML serialization worked completely differently in JDK 7 vs JDK 8 thus the services they make don't quite communicate properly, etc.

[grg0]

> sounds wrong

It's not wrong at all, literally Conway's law: https://en.wikipedia.org/wiki/Conway's_law

[9rx]

> Sounds wrong

Sounds right, no? Service is what people provide, implied in the scope of a macro economy. Microservice then implies the same type of service, but within the micro economy of a single business.

[stephen]

> Microservices are a design pattern for organisations as opposed > to technology ... breakout into multiple teams

I agree, but just saying "multiple teams" has led many eng directors to think "I have two squads now --> omg they cannot both be in the same monolith".

When both squads are 5 people each.

And the squads re-org (or "right size") every 9 months to re-prioritize on the latest features.

Five years go by, 7 team/re-org changes, all of which made sense, but thank god we didn't microservice on the 2nd/3rd/4th/5th/6th team boundaries. :grimmacing:

We should stay "stable, long-lived teams" -- like you need to have a team that exists with the same ownership and mandate for ~18 months to prove its a stable entity worth forming your architecture around.