I'm not necessarily concerned that I will be entirely without a job because of AI. What I am concerned with is that the aspects of my job that I derive pleasure from will be replaced by AI.
So far (super early days, I know) the main thing I've used AI for at work is to write tests for code I'm working on. I get to do the fun bit, solving the problem space, and the AI gets to do the tedious bit (and do it faster than I would). I'm liking at least that level of AI integration.
This is the one that is most scary to me as a user of software. Have the AI write some of the code, sure. But tests have to be correct. Autogenerating a mountain of mostly-correct tests strikes me as a great way of ending up with surprising behavior that is a nightmare to untangle.
From my experience, ChatGPT-3 has been pretty good at exercising all meaningful branches (I look at code coverage results, I don't care for percentages at all), in the least amount of tests, on the first go. I definitely have to modify each test quite a bit, because it frequently hallucinates API calls that don't exist, but the code that it produces is an incredible blueprint. And I haven't even attempted to use RCI yet: "improve your answer" or "your answer is wrong because..."; ChatGPT-4 is supposedly extremely adept at reflecting on its responses. I can only imagine where this will be in a few years.
I was about 6 months late to Copilot because I was incredibly skeptic about it, without having used it in anger. My skepticism was mostly (but not entirely) wrong. Having actually used ChatGPT in anger, I find the degree of skepticism extremely skeptical.
It's like picking up C in the 1970s. We're at the very beginning when things are pretty rough, but the skills that I am building today are going to be foundational in the future. If you're dismissing AI without giving it a few weeks to earn its keep, it's going to be rough to catch up when things improve to the point where it is required.
If you write a unit test for each branch in your method and just put in the current behavior as the expectation, all you've done is created a test that says "the method does what it currently does."
It isn't that stupid, and that was done at least a decade ago with static+control flow analysis. As for AI, I was recently writing tests for a VT push parser in Rust (which is novel code, so no parroting here) and it clearly knew enough about VT to write a, correctly, failing test. I had a bug in my parser, and the test that AI generated found it.
At the end of the day, I'm not sure why anyone would believe the critique of someone who hasn't used a tool in earnest.
That's not necessarily true. For instance snapshot testing, although there may be edge cases missed, the cases cannot be correct or incorrect.
Or you could have the AI write test cases (writing them out is often the most laborious part) and then validate them by hand. That'd be little different than writing them yourself, though again edge cases may be missed. You just skip the un-fun part of repetitively typing out the code for each case.
Tests are easy to verify - if they aren't they are probably bad tests. Using AI for stuff that's easy to verify has been good so far.
Knowing what tests to write is another story.
My problem with AI generated tests is that they lead to over testing and it bogs you down once you have to do refactoring. Ideally detailed tests should come once you're on like 3rd iteration and really sure you've nailed the design (oh I hate TDD if it isn't obvious). With AI I'm getting detailed tests and first try implementations, bad code locked in everywere :(
More tests != better code, tests are still code - the less code you have to satisfy some goal the better.
I see the assumption that automation means it removes the human from the task. The article is precisely about a human is still intermediating. Writing tests doesn’t mean blindly accepting the test, it mean the boilerplate of the test is auto completed based on a human description, then a human verifies the correctness of the output. Will people take the shortcut? Of course. But AI didn’t create or magnify that problem. How many “return true” or incomplete tests or missing significant cases etc have you reviewed or stumbled across? Or worse; the test free code base?
It seems that the more useful way to use AI would be TDD, where you write the tests and perhaps a simple implementation, and AI makes the implementation good/fast and points out inconsistency in the tests.
Of course I validate the tests. I get it to churn them out then go over them to make sure everything is sane and covered as expected. I'm surprised that people would assume otherwise.
Having come from a background of very bespoke tests for systems level software where conventional unit testing just doesn't really work for 90% of cases, I have used ChatGPT to learn how to and quickly write a large test suite for a web-API project I am working on. While ChatGPT has been generally crap at producing anything I would consider committing to my repo, it has worked wonders as a rough guide and a motivation to getting the tests written. Can't be bothered to come up with test ideas? Ask ChatGPT, then I end up writing about twice as many tests anyway, just from getting that initial push.
This is important. Our relationship with those who pay is for labor is indeed “sold” as “do something you love and …” and frankly that does help people over time.
I love coding. Enough that I don’t want to age into being a director/manager, etc.
But what is important to understand is that from the other direction the owners of business only care about profit - they don’t care abt your love for parts of your work, except when they can gaslight and use it to keep you working. For them.
Remember when Dwight Scrute got 13 “employee of the month” awards in a single year because he got 2 in February in lieu of a raise?
That’s what I mean.
AI will be used by owners to either gaslight your love into using it for them, or possible remove what you love to be replaced by AI.
Which, for the record, is not “intelligent” any more than anything else in code.
Anything implemented is either to gaslight our love for this industry for profit, or to replace parts for profit.
You cannot in good conscience be wishing to do menial tasks for a living, that AI can do for you and instead of you.
That's how doing manual search and replace instead of search/replace in editor or computing stuff in spreadsheet by hand instead of using formulas.
If you think AI's output is suboptimal, right. Maybe find a way to use/train AI better to produce good enough output. But simply wishing to out-robot a robot is pointless.
I think the parent's point is the exact opposite of what you have argued against. The author is saying they are afraid 'robots' will come for the fun stuff, rather than only being a tool for the menial labour.
Think of the recent generative AI artwork...rather than drawing, and creating, (some/most?) artists fear just being prompt-carvers. The fun of the art has been replaced with menial labour of crafting prompts.
I wonder how "artsy" kids growing up today feel. Getting good at visual arts takes a lot of time and practice and it must be very discouraging when your best output is easily beaten by a guy typing "some flowers in a vase in the style of Van Gogh" into Dall-E.
You can't expect to be paid for fun stuff if it's done by robots better, until you are looking for artisanal craft career.
Artsy types doing art using basic tools such as brushes will be the same as handicrafted assembly in programming. Rarely needed and not much people are hired to do so.
Fearful of becoming a prompt carver? Let's be real, that is the least of their fears.
Fearful that a prompt carver can replace 10 artists is the more realistic fear here. That is the essential trajectory being plotted out by this technology and the trajectory described by the article.
I enjoy learning about the design philosophy behind an API, or diving into a library to learn the "off the beaten path" workflows. That's where the magic optimizations or shortcuts live.
So far, copilot just shows me the happiest path options for everything I've tried, and really falls over when it's not something that "Well this API looks like API_functionCall, so your query for <get X> must look like API_getX". It rarely does.
Slightly related - there was a Reddit post about a person lamenting the changes in their job. [0] They make 3D models and they greatly enjoyed making models from scratch and taking 2-3 weeks to do so.
Now with AI, they use tools to create a model - and then spend 2-3 days to update it.
This is a productivity improvement, but it also took away a key thing they loved about their job. There isn't really a solution here at all, and the poster wasn't looking for solutions either. Just venting.
We'll be seeing this more in the future, and I wouldn't be surprised if multiple people change careers over it.
Part of me can definitely sympathize with this feeling. Part of me thinks "lol our job is to provide business value using the tool that is software. Not 'getting enjoyment' out of manually twiddling bits." This is just a new layer of abstraction - I'm sure all the old school computer engineers felt slighted when the new software guys didnt even have to know how to solder flipflops and transitors together.