It does seem somewhat hard to imagine being employed as a SWE in 5 years without being proficient in incorporating some sort of generative AI into your work.
This will linger in the courts for years if not a decade, all the while the tech and industry will move on. Think Oracle vs Google case involving Java on Android. Was finally settled but didn't have any impact.
And if it sinks one model, there will be others. Like has been said, the genie is out of the bottle.
Or heaven for bid the AI decides to add one of Microsoft's famous "please wait while we try to fix the problem" dialogs in your project. Especially since I am pretty sure that code looks something like this at a high level
double sleep = random time between now and heat death of the universe
thread.sleep(sleep)
throw prompt that problem couldn't be solved
It's questionable that there will be many jobs that are very similar at all to SWE of today in five years.
There is no reason to believe that the large (language/multimodal) models will stop progressing. In six months the IQ of ChatGPT increased by 30-40 points. If we pretend that these things are linear, (so far they are actually exponential) then that's a 430 IQ in five years. Assuming the acceleration of progress entirely stops and just increases linearly from here on out.
There are quite a few people that think this stuff can easily become millions of times smarter. I am doubtful of that in the short term, because I think there are practical limits to compression, but you have to admit that multiple orders of magnitude increases in speed, data size, and efficiency are very plausible. Especially as new compute-in-memory systems start to roll out which is feasible in five years.
The non-conservative and increasingly popular belief is that these systems will be hundreds of times faster than humans and fully autonomous within a few years. Which seems extremely foolhardy but also the most likely path due to the inability of society to recognize and adapt to the real danger with enough nuance.
So I am wondering if there will be any jobs for any people at that point unless the AI is deeply integrated into their brain.
The only counterpoint I have is that it seems the gpt models have trained on most available information on the Internet. So what’s the future progress there exactly.
Let's suppose that the datasets don't increase in size. Look at the difference in speed between the old ChatGPT and turbo ChatGPT. Suppose within 6-8 months they can do that again with GPT4.
I think that would be about five times faster than a human.
But there is more data. Do you really think they have ingested every single YouTube video or movie? Video and video+transcription is the next thing.
Another source of data could be to have the models study and reprocess information into more concise language (possibly new vocabulary) or diagrams with the goal of increasing levels of abstraction or information density.
Small things like passing new questions into the AI to check for novelty would be nice. Now, if they could find a way to flag and de-rank obviously out-of-date solutions, that would be cool.
If ChatGPT is trained on StackOverflow and people are using ChatGPT instead of StackOverflow, it seems pretty clear ChatGPT's use of StackOverflow's data is not protected under fair use.
there, that was easy to imagine