[cubefox]

On the other hand, often a "quick and dirty" solution unexpectedly becomes the foundation of a lot of other stuff, which can be a persistent pain in the future, while the cost of rewriting the whole thing increases the more other stuff depends on it.

There are two poles: On the one side is making everything an unmaintainable pile of quick hacks, on the other side is over engineering everything. You want to be somewhere in the middle.

[sigseg1v]

Totally agree, it needs a careful balance. But is it possible to know when to use what?

Anecdote: A couple years ago I migrated a 1M+ LOC application from one database to another, cutting the cloud hosting costs practically in half. This was absolutely massive for the company. It took a bit less than a year, but was possible because the people who designed the application didn't write any hardcoded queries, and everything went through an ORM or a query builder. Everything was built to an interface, even though they all only had one implementation. This turned out to be absolutely critical when we had to add the second implementation (and because we needed to maintain compatibility for the original implementation at the same time due to some one-off contracts) and the migration would not have been worth it cost-wise if the proper architecture wasn't in place from the start.

Now take the same application. It has tons of complicated architecture stuff implemented in house. Parts of it are definitely overengineered. It's been apparent when doing other upgrades and finding out "oh... this doesn't work because it's using a custom JSON serializer" that the choice to do that was poor.

In the end, I think the right choice for that application was the complex design that allowed those big migrations. For others, they might never do that and only get hit with downsides.

What it needs is likely a good vision and someone smart to know what to abstract and what to not abstract such that it will have been a good choice 15 years down the road.

[mobiuscog]

ORMs and query builders can also constrain the implementation from a performance and flexibility perspective though.

Your anecdote and reflections are all absolutely true in my experience, but I've found just as many problems with off-the-shelf ORMs.

The key is the clean boundaries/interfaces and keeping things simple - leaving the bridge intact so that your enemies may retreat.

[stouset]

In my 25-year career I’ve experienced easily fifty times as much pain from underthought and hastily-implemented systems than I have from overengineered ones.

Sure, build user-facing stuff cheaply and ready to be thrown away as requirements change. But libraries, tooling, and infrastructure that everything else is built on top of should have thought and care invested in it. This will return dividends.

We spend so much time as an industry worried that somebody might put five percent more effort into a project than it’s worth, that the overwhelming majority of engineers have no idea whatsoever of how to build something to be reliable, to be easily iterated on, and to last. And so we spend tens of times more effort trying to build on top of a mountain of crap.

[dessimus]

> On the other hand, often a "quick and dirty" solution unexpectedly becomes the foundation of a lot of other stuff

This is how Access "databases" and Excel spreadsheets often become mission-critical in SMBs. "We'll deal with needing a database server down the road..."

[wtetzner]

> There are two poles

I'm not sure I agree. I think you can write a simple-but-not-hacky implementation that does only what is necessary, and is also easy to maintain and extend later.

Of course, you never really achieve that ideal, but you can keep getting closer to it as you gain more experience.

[cubefox]

"Easy to extend" arguably means including more abstraction and making things more general than they need to be merely for solving the current problem. There is a trade-off here.

[wtetzner]

"Easy to extend" for me means keeping things decoupled (as opposed to adding extension mechanisms etc.).

Of course there's a tradeoff (there always is), I think my point was just that this option doesn't really seem like it fits between "quick and dirty" and "over engineered", though I suppose that probably depends on how you picture that continuum ;)