Hacker News new | ask | show | jobs
by TheCapn 1060 days ago
Huh. Finally a name for it.

I do a lot of support work for Control Systems. It isn't unheard to find a chunk of PLC code that treats some sort of physical equipment in a unique way that unintentionally creates problems. I like to parrot a line I heard elsewhere: "Every time Software is used to fix a [Electrical/Mechanical] problem, a Gremlin is born".

But often enough when I find a root cause of a bug, or some sort of programmed limitation, the client wants removed. I always refuse until I can find out why that code exists. Nobody puts code in there for no reason, so I need to know why we have a timer, or an override in the first place. Often the answer is the problem it was solving no longer exists, and that's excellent, but for all the times where that code was put there to prevent something from happening and the client had a bunch of staff turnover, the original purpose is lost. Without documentation telling me why it was done that way I'm very cautious to immediately undo someone else's work.

I suppose the other aspect is knowing that I trust my coworkers. They don't (typically) do something for no good reason. If it is in there, it is there for a purpose and I must trust my coworkers to have done their due diligence in the first place. If that trust breaks down then everything becomes more difficult to make decisions on.

18 comments

This is why I comment a "why" for any line of code that's not incredibly obvious. And 100% of the time when it's due to interaction with something outside the codebase, whether that's an OS, filesystem, database, HTTP endpoint, hardware, whatever, if it's not some straightforward call to some API or library.

Sleep due to rate limiting from another service? COMMENT. Who's requiring it, the limits if I know exactly what they are at the time (and noting that I do not and this is just an educated guess that seems to work, if not), what the system behavior might look like if it's exceeded (if I know). Using a database for something trivial the filesystem could plausibly do, but in-fact cannot for very good reasons (say, your only ergonomic way to access the FS the way that you need to, in that environment, results in resource exhaustion via runaway syscalls under load)? Comment. Workaround for a bug in some widely-used library that Ubuntu inexplicably refuses to fix in their LTS release? Comment. That kind of thing.

I have written so very many "yes, I know this sucks, but here's why..." comments.

I also do it when I write code that I know won't do well at larger scale, but can't be bothered to make it more scalable just then, and it doesn't need to be under current expectations (which, 99% of the time, ends up being fine indefinitely). But that may be more about protecting my ego. :-) "Yes, I know this is reading the whole file into memory, but since this is just a batch-job program with an infrequent and predictable invocation and this file is expected to be smallish... whatever. If you're running out of memory, maybe start debugging here. If you're turning this into something invoked on-demand, maybe rewrite this." At least they know I knew it'd break, LOL.

You're doing the lords work. I often get pushback on doing this with some variation of "comments bad, code should be self-documenting". This is unwise, because there are "what code does" and "why code does" comments, but this turns out to be to nuanced to battle the meme.
“Why” isn’t a property of the code itself, though; it’s rather a property of the process of creating the code.

Because of this, my personal belief is that the justification for any line of code belongs in the SCM commit message that introduced the code. `git blame` should ultimately take you to a clear, stand-alone explanation of “why” — one that, as a bonus, can be much longer than would be tolerated inline in the code itself.

Of course, I’m probably pretty unusual, in that I frequently write paragraphs of commit message for single-line changes if they’re sufficiently non-obvious; and I also believe that the best way to get to know a codebase isn’t to look at it in its current gestalt form, but rather to read its commit history forward from the beginning. (And that if other programmers also did that, maybe we’d all write better commit messages, for our collective future selves.)

I think there's room for the comments having a brief explanation and a longer one in the commit message. Sometimes people need the callout to go looking at the history because otherwise they might not realize that there's anything significant there at all.
The "brief explanation" is what the commit message's subject line is for :) Just turn on "show `git blame` beside code" in your IDE, and you'll get all the brief explanations you could ever want!

...but no, to be serious for a moment: this isn't really a workable idea as-is, but it could be. It needs discoverability — mostly in not just throwing noise at you, so that you'll actually pay attention to the signal. If there was some way for your text editor or IDE to not show you all the `git blame` subject lines, but just the ones that were somehow "marked as non-obvious" at commit time, then we could really have something here.

Personally, I think commit messages aren't structured enough. Github had the great idea of enabling PR code-reviewers to select line ranges and annotate them to point out problems. But there's no equivalent mechanism (in Github, or in git, or in any IDE I know of) for annotating the code "at commit time" to explain what you did out-of-band of the code itself, in a way that ends up in the commit message.

In my imagination, text-editors and IDEs would work together with SCMs to establish a standard for "embeddable commit-message code annotations." Rather than the commit being one long block of text, `git commit -p` (or equivalent IDE porcelain) would take you through your staged hunks like `git reset -p` does; but for each hunk, it would ask you to populate a few form fields. You'd give the hunk a (log-scaled) rating of non-obviousness, an arbitrary set of /[A-Z]+/ tags (think "TODO" or "XXX"), an eye-catching one-liner start to the explanation, and then as much additional free-text explanation as you like. All the per-hunk annotations would then get baked into a markdown-like microformat that embeds into the commit message, that text-editors/IDEs could recognize and pull back out of the commit message for display.

And then, your text-editor or IDE would:

1. embed each hunk's annotation-block above the code it references (as long as the code still exists to be referenced — think of it as vertical, hunk-wise "show `git blame` beside code");

2. calculate a visibility score for each annotation-block based on a project-config-file-based, but user-overridable arbitrary formula involving the non-obviousness value, the tags, the file path, and the lexical identifier path from the syntax highlighter (the same lexical-identifier-path modern `git diff` gives as a context line for each diff-hunk);

3a. if the visibility score is > 1, then show the full annotation-block for the hunk by default;

3b. else, if the visibility score is > 0, then show the annotation-block folded to just its first line;

3c. else, hide the annotation-block (but you can still reveal the matching annotation with some hotkey when the annotated lines are focused.)

Of course, because this is just sourced from (let's-pretend-it's-immutable) git history, these annotation-block lines would be "virtual" — i.e. they'd be read-only, and wouldn't have line-numbers in the editor. If the text-editor wants to be fancy, they could even be rendered in a little sticky-note callout box, and could default to rendering in a proportional-width font with soft wrapping. Think of some hybrid of regular doc-comments, and the editing annotations in Office/Google Docs.

---

...though, that's still not going as far as I'd really like. My real wish (that I don't expect to ever really happen) is for us to all be writing code as a series of literate codebase migrations — where your editor shows you the migration you're editing on the left, and the gestalt codebase that's generated as a result of running all migrations up to that one on the right, with the changes from the migration highlighted. You never directly edit the gestalt codebase; you only edit the migrations. And the migrations are what get committed to source-control — meaning that any code comments are there to be the literate documentation for the changes themselves; while the commit messages exist only to document the meta-level "editing process" that justifies the inclusion of the change.

Why? Because the goal is to structure the codebase for reading. Such a codebase would have one definitive way to learn it: just read the migrations like a book, front to back; watching the corresponding generated code evolve with each migration. If you're only interested in one component, then filter for just the migrations that make changes to it (`git log -S` style) and then read those. And if you, during development, realize you've just made a simple typo, or that you wrote some confusing code and later came up with a simpler way to express the same semantics — i.e. if you come up with code that "you wish you could have written from the start" — then you should go back and modify the earlier migration so that it's introduced there, so that new dev reading the code never has to see you introduce the bad version and then correct it, but instead just sees the good version from the start.

In other words: don't think of it as "meticulously grooming a commit history"; instead, think of it as your actual job not being "developer", but rather, as you (and all your coworkers) being the writers and editors of a programming textbook about the process of building program X... which happens to compile to program X.

I don't understand the focus on commit messages. I never read the git log. You can't assume anyone reading the code has access to the commit history or the time to read it. The codebase itself should contain any important documentation.
One could accomplish a part of your vision, today, by splitting commits into smaller commits. Then you have the relation between hunks of changes and text in the commit message on a smaller level. Then you can use branches with a non-trivial commit message in the merge commit to document the whole set of commits.

As far as I know changes to the Linux kernel are usally submitted as a series of patches, i.e. a sequence of commits. I.e. a branch, although it is usually not represented as git branch while submitting.

I disagree.

WHY: Using _______ sort because at time of writing code, the anticipated sets are ... and given this programming language and environment ... this tends to be more performant (or this code was easiest and quickest to write and understand).

This way when someone later come along and says WTF?! They know why or at least some of the original developers reasoning for choosing that code implementation.

Completely agree. I find this attitude toward (against?) comments is common in fields where code churn is the norm and time spent at a company rarely exceeds a year or two. When you have to work on something that lasts more than a few seasons, or are providing an API, comments are gold.
Concurrence: the ability of the code to be self-documenting ends at the borders of the code itself. Factors outside of the code that impose requirements on the code must be documented more conventionally. The "sleep 10 seconds to (hopefully) ensure that a file has finished downloading" anecdote from upthread is a great example.
I indeed enjoyed.
Self-documenting code is perfectly capable of expressing the "why" in addition to the "what". It's just that often the extra effort and/or complexity required to express the "why" through code is not worth it when a simple comment would suffice.
> Self-documenting code is perfectly capable of expressing the "why" in addition to the "what".

I don't think anymore that's true, at least in a number of areas.

In another life, I've worked on concurrent data structures in java and/or lock-free java code I'd at this point call too arcane to ever write. The code ended up looking deceptively simple and it was the correct, minimal set of code to write. I don't see any way to express the correctness reasoning for these parts of code in code.

And now I'm dealing with configuration management. Code managing applications and systems. Some of these applications just resist any obvious or intuitive approach, and some others exhibit destructive or catastrophic behavior if approached with an intuitive approach. Again, how to do this in code? The working code is the smallest, most robust set of code I can setup to work around the madness. But why this is the least horrible way to approach this, I cannot express that in code. I can express this in comments.

This is a statement which is technically-true (so long as your language of choice has no length on names), but unhelpful since it does not apply in most practical cases.
I like to make sure the "why" is documented, but it's hard to get people to care about that.

I remember a former client tracking me down to ask about a bug that they had struggled to fix for months. There was a comment that I'd left 10 years earlier saying that while the logic was confusing, there was a good reason it was like that. Another developer had come along and commented out the line of code, leaving a comment saying that it was confusing!

Hah, I did one of these just last week. There's some sort of silicon bug or incorrect documentation that causes this lithium battery charger to read the charge current at half of what it should be. This could cause the battery to literally explode, so I left a big comment with lots of warnings explaining why I'm sending "incorrect" config values to the charge controller.

It's absolutely imperative that the next guy knows what the fuck I'm doing by tampering with safety limits.

I like to add the date the why comment was added and the date the comment was last reviewed/veified as still true/neccesary (which will rarely differ because they are seldomly reviewed/re-verified).

COMMENT WRITTEN: 2023-03-21

COMMENT LAST REVIEWED/VERIFIED AS STILL TRUE: 2023-05-04

WHY THIS CODE: This sucks but ...

There was some inscrutable code that had sleep 10 [seconds] in the middle.

It took several hours to figure it out, but the sleep was there in case a file had not finished downloading

I see your "wait for a file to finish downloading", and raise you a "wait before responding because, for one of our clients, if the latency is below a certain threshold, it will assume that the response was a failure and will discard it". That was a fun codebase.
Back in the 90s, we were trying to debug a crash on a customer’s site (using the then miraculous pcAnywhere). We couldn’t figure it out, and in desperation sent a developer that lived in the same country to observe ‘live’. As soon as he watched us reproduce it, he said “the modem was still disconnecting” - he could hear it, and we couldn’t. The solution, of course, was a sleep statement.
shudder
I like to write frequent comments, so that I can just scan the comments (only) of a function to know how and why it does what it does.
The last part is also useful. Tells the next person where to look when they do run into scaling issues, and also tells them that there wasn't some other reason to do it.
> But often enough when I find a root cause of a bug, or some sort of programmed limitation, the client wants removed. I always refuse until I can find out why that code exists. Nobody puts code in there for no reason, so I need to know why we have a timer, or an override in the first place.

Isn't that just the regular Chesterton's Fence argument though?

The one the article is specifically written to point out is not enough by itself, because you need to know what else has been built with the assumption that that code is there?

All my comment is adding a software anecdote to the story. It really is just regular Chesterton's Fence, a term I've never heard until now but dealt with for the last several years.

You're not wrong, but in the context of a PLC controlling a motor or gate it is far more segregated than the code you're probably thinking of. Having a timer override on a single gate's position limit sensor would have no effect on a separate sensor/gate/motor.

If the gate's function block had specific code built into it that affected all gates then what you're talking about would be more applicable.

I think they’re thinking of things like that software control for the health machine that could get in a state where it would give a lethal dose to a patient.
The most haunting comment line I've ever seen was buried deep in an Allen Bradley PLC:

> I don't know why this rung is needed but delete it and see what happens for yourself

Did not fuck around; did not find out.

Context for those who haven't worked in the field: A PLC is a programmable logic controller. They are typically programmed with ladder logic which grew out of discrete relay based control systems.

Generally they're controlling industrial equipment of some sort, and making changes without a thorough understanding of what's happening now and how your change will affect the equipment and process is frowned upon.

https://en.wikipedia.org/wiki/Ladder_logic

I interned briefly at a company which mainly built industrial control systems. One of its most interesting features (which is also very mind-bending if you're coming from any sort of typical programming ecosystem) is that every "rung" is evaluated in parallel. (As a physical relay-based control system would have back in the day.)

Very wild stuff.

This has the "more magic" anecdote feel to it.

http://www.catb.org/jargon/html/magic-story.html

I remember reading a story like this from the early days of Acorn. The first production sample of the BBC Micro came in, and would crash unexpectedly. Trial and error found that connecting a jumper wire between 2 particular points on the board stopped it crashing, but nobody could work out why it crashed or how that fixed it. They never worked it out and ended up shipping mass quantities of the BBC Micro with the magic jumper wire in place on each one.
but now its like the number of licks to the center of tootsie pop the world may never know!
do you remember what the instruction was?
I think it was something like

|---]/[---[ONS]---[MOV]---|

> I do a lot of support work for Control Systems. It isn't unheard to find a chunk of PLC code that treats some sort of physical equipment in a unique way that unintentionally creates problems. I like to parrot a line I heard elsewhere: "Every time Software to fix a [Electrical/Mechanical] problem, a Gremlin is born".

At least some of this is cultural. EEs and MEs have historically viewed software less seriously than electrical and mechanical systems. As a result, engineering cultures dominated by EEs/MEs tend to produce shit code. Inexcusably incompetent software engineering remains common among ostensibly Professional Engineers.

You're not wrong. It shows in the state of PLC/HMI Development tools. Even simple things like Revision Control is decades behind in some cases.

I've basically found my niche in the industry as a Software Engineer though I can't say I see myself staying in the industry much longer. The amount of time's I've gotten my hands on code published by my EE coworkers only to rewrite it to work 10x faster at half the size with less bugs? Yikes. HMI/PLC work is almost like working in C++ at times, there's so many potential pitfalls for people that don't really understand the entire system, but the mentality by EE/ME types in the industry is to treat the software as a second class citizen.

Even the clients treat their IT/OT systems that way. A production mill has super strict service intervals with very defined procedures to make sure there is NO downtime to production. But get the very same management team to build a redundant SCADA server? Or even have them schedule regular reboots? God no.

I have no clue, but I wonder how unique this attitude turns out to be among MEs/EEs versus, say, everyone dealing with electronics. Because the stories and complaints about mechanical and electrical engineers treating code (and programmers) as second-class, remind me very much of how front-end people complain about how they’re perceived by back-end people. And about how backend people complain about how they’re perceived by systems programming people. And so on up/down the line(s) of abstraction.
> It shows in the state of PLC/HMI Development tools.

i mean you are talking about upgrading things that are going to be in service for decades perhaps. the requirements for the programs is generally not complicated. turn on a pump for a time, weigh something and then alarm if some sensor doesn't see something.

Structured text was an improvement over ladder logic as you could fit more of the particular program in the screen real estate you had and could edit it easier since it was just text. though, that had its own set of issues that needed to be worked through and it wasn't a panacea.

I think one of the reasons I've seen this happen is because typically, EE and ME programs in university teach very little CS "enough to be dangerous", and the few coding projects you are required to do are often taught in a way that downplays the importance of the software. Software is often seen as simply a translation or manifestation of a classical mathematical model or control system ( or even directly generated by Matlab/Simulink).

Software, being less familiar, is not viewed as a fundamental architectural component because there often isn't sufficient understanding of the structure or nuance involved in building it. In my experiences software or firmware engineers tend to be isolated from the people who designed the physical systems, and a lot of meaning is lost between the two teams because the software side does not understand the limitations and principles of the hardware and the hardware team does not understand the capabilities and limitations of the software.

The worst part is there's no particular reason for this -- infusing proper software development best practices into existing EE/ME coursework isn't that hard.

It's an instance of the larger pattern in which technical degree programs lag industry requirements by decades, as older faculty ossify at the state of the art circa 2-3 years prior to when they received tenure.

IMO one way to help would be to get rid of the entire notion of a "Professor".

Instead, courses should be taught primarily by a combination of professional instructors on permanent contracts and teacher-practitioners supported by the instructors. The instructors should have occasional sabbaticals for the professional instructors to embed in firms and ensure they're up to date on the industry.

The research side of the university can even more easily replace Professors and tenure with first-line lab managers on 3-5 year contracts whose job is simply to apply for grants and run labs, and who can teach if they want but are held to the same standards as any other applicant for an ad junct teaching position in any particular term.

I definitely think there are two sides to this. The school I went to had a lot of professors for whom the "ossified 2 years prior to tenure" thing was true for, but I also found them to be helpful for teaching fundamental concepts that didn't change in an effective way.

I think one barrier to better engineering programs in universities is that there typically is an onerous set of "accreditation requirements" which prevents significant modification of the curriculum to adapt to modern needs.

The other barrier is that students culturally appear to not always want to do more coding than needed. Courses involving coding were widely regarded as the most difficult by the people around me, despite something like up to 80% of an EE class going into SW engineering after graduating.

I think in general, degree programs are designed to be something that they're not used for anymore often. The usual line is that they're designed to provide a track to academia, and aren't vocational training. But nowadays degrees seem very ritualistic and ornamental - it seems that people are doing their learning on the job mostly whatever they do, and the relevance of the degree itself is just a shibboleth of some sort.

> I think one barrier to better engineering programs in universities is that there typically is an onerous set of "accreditation requirements" which prevents significant modification of the curriculum to adapt to modern needs.

This seems to be rapidly dissolving, at least in California. Several schools including Stanford, Cal Tech, and several of the UCs have dropped ABET accreditation for most of their programs in recent years, with more likely to follow as they come up for renewal.

Why should EEs also be software engineers? These are two distinct specializations.

No sane person would expect a programmer to just design a lithium battery charge circuit that goes in your user's pocket, that'd be reckless and dangerous. I likewise would never expect a programmer to break out an oscilloscope to debug a clock glitch, or characterize a power supply, or debug protocol communication one bit at a time. I wouldn't ask a programmer to do FCC pre-validation, or to even know what that means.

Why then do you want to rely on an EE to produce critical software?

As an EE, I know my limits and how to separate concerns. I keep my bad C++ restricted to firmware and I simply do not write code further up the stack. We have programmers for that. Where the app needs to talk to the device, I'll write an example implementation, document it, and pass it off to the programmer in charge. It's their job to write good code, my job is to keep the product from catching fire.

If you want good code, hire a programmer. If you want pretty firmware, hire a programmer to work with your EEs. If you expect an EE to write code, you get EE code because their specialization is in electronics.

Unless you really want an EE who is also a software engineer, but then you're paying someone very highly skilled to do two jobs at once.

Electronics and software are two different jobs requiring two differently specialized people. It just looks like they overlap.

I think it is the result of how these things tend to be taught. At least in my school, all the EE’s and all the computer engineering students had the same first couple programming classes.

Lots of EE’s need to do some programming, and lots of people getting EE degrees end up in programming careers, so it would be a disservice not to teach them any programming at all. In particular, an engineer should be taught enough programming to put together a matlab or Numpy program, right?

Meanwhile, some of their classmates will go on to program microcontrollers as their job.

Writing programs and a product, and writing programs to help design a product, are two basically different types of jobs that happen to use similar tools, so it isn’t that surprising that people mix them up.

This is a discussion that is much larger than what's available in a comment section like this. But I agree with you wholeheartedly.

I think part of the thing is Software Engineers haven't been a thing for as long in the industry. I'm the only Software Engineer I've met doing controls. My supervisor has a CS degree and an Electrical Technician diploma, but never another SE.

Second is I think up until recently, the work done by Control Systems has been what's capable of an EE or ME so having a SE hasn't been necessary. I've been with my company for 10 years now, and in that time I've watched the evolution of what my clients are seeking in terms of requirements to their systems.

I primarily work in Agriculture or Food Production. 10 years ago my projects were assembling plants and getting their motors to start, with the required protections then some rudimentary automation to align paths and ensure flow.

Today? I'm building traceability systems to report on exactly which truck load was the origin of contamination for a railcar shipped months later. Or integrating production data to ERP systems. Adding MES capabilities to track downtimes and optimize production usage. Generating tools to do root cause analysis on events... It's a different world and the skills of a Software Engineer haven't really been a super important role for quite a while.

I think the mindset is shifting, but it is slow.

On the other hand there was Juicero...
I don't follow. Did you accidentally reply to the wrong comment?
No, why? The point was EEs and MEs, or rather traditional engineering heavy culture produces bad software (never mind the first software devs tended to be EEs), so Juicero is good example of a software-leaning culture producing shitty hardware products.
Uh, the Juicero was spectacular hardware. The mechanical engineering in that beast was absolutely beautiful.

I don't recall what the software was like, but none of that is why it failed, it was simply a moronic business idea. An overpriced subscription for low quality fruit in a DRM-laden pouch. Nobody wanted it then or now.

The Juicero was terrible hardware in the sense that they could have made a product that was functionally similar but cost far less to make. It seems like they got a hardware engineering team straight out of college and gave them no constraints or budget. You have a giant CNC'd aluminum frame, a custom gearbox, a custom power supply, a custom drive motor, etc. All of this is only necessary because they decided to squeeze the juice out of the bags by applying thousands of pounds of force to the entire surface of the bag at once vs. using a roller or something. They were likely losing hundreds of dollars per unit even when they were selling the press for $600.
I kinda want a Juicero now, but only as an object d'Art. It was a garbage product.
I'll never do PLC work again. Forget undocumented code, most of the time there's no schematics for the hardware you're working on because it was custom-built thirty years ago.
My company is generally good about that. We have lots of overlapping documentation that answers questions like that in different ways. From Electrical schemas to QA docs, picture archives of panels and wiring, ticketing systems, spreadsheets over I/O, etc. etc.

I hate PLC work for other reasons. I'm starting to look at going back to more traditional software role. I'm a bit tired of the road work and find the compensation for the amount asked of you to be drastically underwhelming. This meme is very much relevant:

https://i.redd.it/rawo5uki1v9b1.jpg

Ayep. When you're the PLC guy they expect you to know how to fix anything that plugs into a wall but isn't understood by the electricians.
> I suppose the other aspect is knowing that I trust my coworkers. They don't (typically) do something for no good reason

This so much. Depending on the git blame, I'll either remove it blindly or actually think about it way more.

> "Every time Software is used to fix a [Electrical/Mechanical] problem, a Gremlin is born"

Early in my career, I was confused by seemingly-crazy questions in the Hacker Test (https://www-users.york.ac.uk/~ss44/joke/hacker.htm) like...

> 0133 Ever fix a hardware problem in software?

> 0134 ... Vice versa?

But after spending years developing embedded systems, I don't even blink at such questions. Yes, of course I have committed such necessarily evils!

> Nobody puts code in there for no reason, so I need to know why we have a timer, or an override in the first place.

I would like to think that if I sent out an email about git hygiene that you would support me against the people who don’t understand why I get grumpy at them for commits that are fifty times as long as the commit message, and mix four concerns two of which aren’t mentioned at all.

Git history is useless until you need it, and then it’s priceless.

I can’t always tell what I meant by a block of code I wrote two years ago, let alone what you meant by one you wrote five years ago.

> commits that are fifty times as long as the commit message

One of my proudest commits had a 1:30 commit:message length ratio. The change may have only been ~3 lines, but boy was there a lot of knowledge represented there!

Been there, got the tshirt. The one liners can be the worst. Any time the comment has a ticket url for another product it tends to get juicy.
I work with control systems and have a similar mantra: "You can't overcome physics with software." It's super common to have someone ask if a mechanical/human/electrical/process issue can be fixed with software because people believe that programming time is free. Sometimes it's not even that it's impossible to do in software, but adding unnecessary complexity almost always backfires and you'll wind up fixing it the right way anyway in the end.
Chesterton's Fence is a principle that says change should not be made until the reasoning behind the current state of affairs is understood. It says the rash move, upon coming across a fence, would be to tear it down without understanding why it was put up.
Sometimes you have to remove the chunk and see what breaks because the person that put it in is long gone and documentation is missing
This sounds more like the original Chesterton’s fence than what the article is describing. The article is about understanding something’s actual current purpose, rather than just the intended purpose.

What the article is describing reminds me of the XKCD comic workflow: https://xkcd.com/1172/

A system exists external to the creators original purpose, and can take on purposes that were never intended but naturally evolve. It isn’t enough to say “well that is not in the spec”, because that doesn’t change reality.

The unspoken thing here is that PLC code often(usually?) isn't exactly written in text, or in a format readable by anything other than the PLC programming software.

After a year long foray into the world of PLC, I felt like I was programming in the dark ages.

I'm assuming its a bit better at very big plants/operations, but still.

> "Every time Software is used to fix a [Electrical/Mechanical] problem, a Gremlin is born"

I'm definitely going to use this, and I think there's a more general statement: "Every time software is used to fix a problem in a lower layer (which may also be software), a gremlin is born."

cough 737 Max cough
> "Every time Software to fix a [Electrical/Mechanical] problem, a Gremlin is born".

I think I get the gist, but that sentence is missing some words.

oooboy is it. That's what I get for not proofreading.
> "Every time Software is used to fix a [Electrical/Mechanical] problem, a Gremlin is born"

Gizmo caca.

(...I just watched both Gremlins movies last weekend...)

> "Every time Software is used to fix a [Electrical/Mechanical] problem, a Gremlin is born".

Just did one of those this morning. Hmmm.