[singron]

> You'll have a monolith, it might break out into frontend, backend and a separate service for async background jobs

And when you break these out, you don't actually have to split your code at all. You can deploy your normal monolith with a flag telling it what role to play. The background worker can still run a webserver since it's useful for healthchecks and metrics and the loadbalancer will decide what "roles" get real traffic.

[elevatedastalt]

If you are building the same binary for all microservices you lose the dependency-reduction benefit microservices provide, since your build will still break because of some completely unrelated team's code.

[roguecoder]

If it is possible for that other team to merge a broken build, you are doing it wrong.

If you are concerned about someone else breaking your thing, good! You were going to eventually break it yourself. Write whatever testing gives you confidence that someone else's changes won't break your code, and, bonus, now you can make changes without breaking your code.

[motorest]

> If it is possible for that other team to merge a broken build, you are doing it wrong.

This assertion is unrealistic and fails to address the problem. The fact that builds can and do break is a very mundane fact of life. There are whole job classes dedicated to mitigate the problems caused by broken builds, and here you are accusing others of doing things wrong. You cannot hide this away by trying to shame software developers for doing things that software developers do.

> Write whatever testing gives you confidence that someone else's changes won't break your code, and, bonus, now you can make changes without breaking your code.

That observation is so naive that casts doubt on whether you have any professional experience developing software. There are a myriad of ways any commit can break something that goes well beyond whether it compiles or not. Why do you think that companies, including FANGs, still hire small armies of QAs to manually verify if things still work once deployed? Is everyone around you doing things wrong, and you're the only beacon of hope? Unreal.

[roguecoder]

I haven't seen a broken build in at least nine years, not since I left the company with a merge process built out of bash scripts that took three hours and required manual hand-holding.

I am genuinely curious what situations you are seeing where builds are making it through CI and then don't compile.

It isn't always worth investing in quality, but when it is it is entirely possible to write essentially bug-free software. I've gone seven months without a bug in production and the one we saw we had a signed letter from product saying "I am okay if this feature breaks, because I think writing the tests that can verify this integration was going to take too long."

FAANG companies aren't prioritizing writing software well: they are prioritizing managing 50,000 engineers. Which is a much harder problem, but the management solutions that work for that preclude the techniques that let us write bug-free software.

One of the great things about startups is that it is trivial to manage five engineers, so there is no reason we have to write software badly.

[motorest]

> (...) it is entirely possible to write essentially bug-free software (...)

You lost what little credibility you had left.

[JackSlateur]

You are absolutely right.

Of course, if people wrote bug-free code, then there would be no bug !

Bug-free code in the actual code, or bug-free code in the test code, this is the same story.

If you write stuff and never have any bug, then either:

  - you are lying
  - you do not write much
  - you only write really simple things
  - you are Jesus, came back from heaven to shine his light on us, poor souls

The more complicated, intricate stuff you have, the more bugs you'll get (and only time will allow you to fix that).

Tests are great do define how you think it should work, and to ensure it keeps doing that way. Take the time to think about the third point on the bullet list above.

[quesera]

This seems a bit much.

In the DVCS era, we have inexpensive branching. Do as thou wilt on your topic or epic branches. Rebase them against main/master before merging upwards. Fix what must be fixed first.

Main/master branch should never fail CI. If it does, there is something seriously wrong with your branch lifecycle and/or deployment process.

[Seb-C]

I disagree. I write complex code, and it is essentially bug-free.

And no, I'm not Jesus, I just care a lot about quality and have spent the last 20 years finding ways and strategies to improve it.

Reducing the number of bugs does not mean being a god that writes bug-free code on the first draft. It means being able to detect and fix issues as early as possible. In my case I aim to always do that before letting myself push any code to git.

IMO, it only comes down to how much someone really cares about the quality, but here are some examples of what can be done and is very effective:

- Plan ahead your functional and technical design

- Carefully research existing code to confirm the feasibility of the design

- Use a statically typed language

- Use advanced static-analysis tools

- Avoid magic, write explicit code. Especially avoid runtime checks such as reflection. Ideally, everything should be checked statically one way or another.

- Never let a code path/branch/corner case be unhandled, however unlikely it is (and go back to step one to refine the design if a code path has been forgotten in the current design)

- Always have automated testing. The bare minimum is to unit-test all business logic, including all possible code paths. Ideally e2e tests are nice, but not always a good investment. Tests must be 100% independent and never depend on an external environment, otherwise it's going to be flaky at some point.

- Always manually test every feature and path related to my changes (especially don't skip testing the ones that I think are going to be ok) before pushing anything to git.

- Warnings and "optional" notices are unacceptable and must always be fixed (or disabled), otherwise the list will just keep growing, which reduces the visibility of any issue and normalizes having problems.

- Have a CI integration that applies all the automated checks mentioned in this list and make everything mandatory.

Each one of those actions does on it's own significantly reduce the number of bugs. If you combine them all, you can effectively reduce the number of bugs to pretty-much zero. And since the earlier you find a bug, the cheaper is it overall to fix, I've also found-out that in terms of productivity it's always worth the investment (despite many people pretending the opposite).

[tuckerman]

Even if it builds successfully, I've never worked anywhere where automated tests prevented 100% of problems and I doubt I ever will. For most systems of sufficient complexity you are testing in prod, even if you did a lot of testing before prod as well.

[roguecoder]

That's even more true for microservices, though, since I have yet to see a microservice architecture that automatically runs end to end tests before deploying.

The post I was replying to said "your build will still break": that's what I was taking issue with. In this day and age there is no reason our trunk build should ever be broken.

[mjr00]

> I have yet to see a microservice architecture that automatically runs end to end tests before deploying.

One of the big tenets of independent services is that your APIs are contracts that don't change behaviour. As long as each individual service doesn't introduce breaking changes, the system as a whole should work as expected. If it doesn't this is indicative of either 1) a specific service lacking test coverage, or 2) doing something wrong i.e. directly reading from a microservices' database without going through an API.

[triceratops]

> One of the big tenets of independent services is that your APIs are contracts that don't change behaviour

How is that any different from an API in a monolith not changing behavior?

[tuckerman]

Yes, I suspect some of the back and forth is the fuzziness of the term "broken build", whether that means the code literally doesn't compile or it does but the code does the wrong thing.

I agree that you can prevent merges that cause compilation errors in nearly all cases!

[jimbokun]

What about when it’s you breaking your own thing?

A very large code base full of loosely related functionality makes it more and more likely a change in one part will break another part in unexpected ways.

[DanielHB]

You have a point, but I wouldn't say this is a big deal unless there is a mammoth dependency somewhere that slow down things to a crawl. Then maybe that one part of the codebase can be broken into its own separate service.

But even then there are ways around this kind of problem with dynamic linking pre-built binaries and caching, but it is extra complexity that could be worse than managing multiple services. Docker cache can usually handle this pretty well though.

[jounker]

You’ll still get some isolation since not all pathways share the same code. It’s not all or nothing.