|
We software engineers have a different experience with AI than most other fields. We have vast arrays of guardrails in our field, with unit testing, CI/CD, source control being relatively easy since we mostly produce textual artifacts, strong typing, compilers with decades of experience in giving targeted error messages, all sorts of things [1]. When AI takes two steps forward and then does something stupid, we have so many guardrails that our harnesses can easily get the error message, feed it back in, and the AI can pick up the pieces and carry on. We hardly even notice this process. If you watch an AI and dig into the details of everything it is doing, you can see it repeatedly banging into these guard rails. There's nothing wrong with that necessarily. As a human being, I bang into those guard rails all the time too. That's where they came from in the first place, to let humans bang into them. And we've built a lot of them. However, in a lot of the rest of the world their experience with AI much more resembles that of the more critical voices that post here. A lawyer who uses an AI that makes two correct citations and then an incorrect one has many fewer automated guard rails to work with. It is relatively easy to imagine a system that at least verifies the citation exists (I've heard that's easier said than done due to the system depending on humans to resolve sloppy references but still it's feasible), but the task of deciding if the AI correctly used the citation, either in the abstract sense of it being correct or in the sense of it being the best way to use it to advance the current case, is a vastly harder decision than "ah, that change failed to compile, try again". Accounting seems like another good example. Yes, it has the obvious guard rail of "do the books balance", but that's the beginning of accounting, not the end. It's difficult to put up guard rails for how the accounting is done from there. An accountant will experience an AI accountant as doing OK sometimes but making really dumb decisions that couldn't be caught by anything other than human review, and I have to imagine that the lack of learning and the way the AI will tend to make the same mistakes over and over must be incredibly aggravating. I think there's probably more truth to "AI is useless" than we may see. I think a lot more people than we realize have had the experience of using AI a while, putting some trust in it, then having that trust grotesquely violated when it says something stupid in an email or makes boneheaded errors in a spreadsheet. We're maybe just now exiting the portion of the hype cycle where it is simply culturally unacceptable to criticize the AI and entering the part where it is culturally both acceptable and expected, and we software engineers may look on in bafflement at the other fields and their complaints because it's working for us, what's your problem? [1]: https://jerf.org/iri/post/2026/programming_is_engineering/ |
When I look at the clerical work from a software engineer's point of view, I find it nuts how few guardrails there are - clerical workers would have a far easier time producing work with a low error rate if there were some guardrails put up, but that's just not an investment/culture that the field seems to be interested in.
The two related points around AI are that 1) guardrails would make the clerical work relatively easier for AI and 2) because assistants are so error-prone anyway, the bar for AI isn't all that high.