[redleggedfrog]

The article didn't mention what I have observed as the worst cause of bloat - what I call the "New Toys" problem.

For example, you need a process to export data every 4 hours, with some visibility of success and failures. I could have written a cron job/scheduled task in 4 hours and be done. What I found instead is Kafka with node.js and couch.db. Yes, for that one export. Not only that the were paying monthly for the Kafka. Soooooo, it got replaced.

I've seen this a lot more in the last 10 years. I call them "stitcher programmers." They are near useless at providing solutions unless they can stitch together some byzantine Frankenstein's monster from existing tech, usually with extreme overkill. On the front end is the worst with React and thousands of dependencies for simple forms.

Right sizing a solution is not in their vocabulary.

[Spivak]

Look, I obviously don't know the specifics but I think you swung too far on the other side and also just stitching together existing components. A single box running cron is quick and easy but I would be wary of hinging anything non-dev facing on that.

* You can't fearlessly patch the cron box without knowing when the jobs run, I don't want any special cases. Also how would I even know when you have jobs scheduled looking at a fleet -- read your out of date docs? Ew no. So a messaging system, guess the devs were already familiar with Kafka, is necessary to process the jobs across multiple nodes.

* Individual nodes are unreliable and you don't have any durable persistent storage. Most people don't like storing data in Kafka even though it's possible so they went with a database.

* Cron doesn't have any mechanism to give you a history of jobs that isn't built into your script or parsing logs. Ditto with failure notifications. You also can't reprocess failed jobs except manually. Guess you just wait another 4 hours?

* You now also can't duplicate the server because they're both going to try and do the export every 4 hours and step on one another. Woop, you made a system where an assumed safe operation "adding something" breaks stuff.

This kind of thing is a nightmare if you already have queues and a database because why would you stand up another thing but if you had none of that to begin with then yeah... makes total sense.

Like this is the reality of ops and running something "production ready" it's a lot of big ole complex HA platform so that you can run your 5 lines of code and not have to worry about any of the hard problems like availability, retrying, resource contention, timing, data loss, locking.

[redleggedfrog]

You're proving my point. None of those bullets are even considerations - they're problems in search of money. The initial solution was implemented by someone who didn't bother to notice everything else on this system was using scheduled tasks (or services and Quartz) to do their work. The didn't bother to notice that there was already Serilog setup to do reporting to another system that the customer was already using to monitor processes. They didn't bother to look for the local storage that was available. Instead they purchased a Kafka instance at the company's cost and threw tech at it. Oh, and then quit and got a different job, before writing any docs.

My solution integrated with current tech, didn't have any of the problems outlined in those bullet points, and has required 0 maintenance and or updates for over 4 years. I don't want to even think about how many Kafka and couch releases there have been since then.

[ben7799]

This kind of developer is always trying to bring something into the stack with an ulterior motive of being able to put that new piece of tech into their resume as another bullet point.

It's a HUGE problem.. the devs who do it also tend to be the exact opposite of K.I.S.S. They're always looking for the most complicated and obtuse way of doing anything to make themselves look smart.

[xorcist]

> stitch together some byzantine Frankenstein's monster from existing tech,

Or, you could stich it together from new tech.

It wasn't long ago I was surprised to discover a Hadoop installation at my current professional setting, complete with Zookeepers and everything. After some sleuthwork (which wasn't that bad as they keep everything in git, in a future when everything is api calls to some Kuberentes clusters we're all smoked) it turned out that all it does it move data between two systems.

Literally. It could be replaced with a periodic rsync run. Instead someone has to maintain and monitor a whole suite of software, prepare, test and run upgrades and so on.

[yjftsjthsd-h]

I don't think "stitching" is the problem per se; taking off-the-shelf parts can be a good thing or a bad thing depending on the parts and how they're used. I mean, is there any difference between stitching together cron and a couple shell scripts vs stitching together kafka and node, other than how many components are and how well suited they are to the job and environment? It sounds like your problem is with people who have new-shiny syndrome and try to force their latest interest into their work regardless of whether it makes sense.