[currymj]

I think if you believe LLMs can truly generalize and will be able to replace all labor in entire industries and 10x every year, you pretty much should believe in ASI at which point having a job is the least of your problems.

if you rule out ASI, then that means progress is going to have to slow. consider that programming has been getting more and more automated continually since 1954. so put yourself in a position where what LLMs can do is a complement to what you can do. currently you still need to understand how software works in order to operate one of these things successfully.

[throw234234234]

I don't know if I agree with that and as a SWE myself its tempting to think that - it it a form of coping and hope that we will be all in it together.

However rationally I can see where these models are evolving, and it leads me to think the software industry is on its own here at least in the short/medium term. Code and math, and with math you typically need to know enough about the domain know what abstract concept to ask, so that just leaves coding and software development. Even for non technical people they understand the result they want of code.

You can see it in this announcement - it's all about "code, code, code" and how good they are in "code". This is not by accident. The models are becoming more specialised and the techniques used to improve them beyond standard LLM's are not as general to a wide variety of domains.

We engineers think AI automation is about difficulty and intelligence, but that's only partly true. Its also about whether the engineer has the knowledge on what they want to automate, the training data is accessible and vast, and they even know WHAT data is applicable. This combination of both deep domain skills and AI expertise is actually quite rare which is why every AI CEO wants others to go "vertical" - they want others to do that leg work on their platforms. Even if it eventuates it is rare enough that, if they automate, will automate a LOT slower not at the deltas of a new model every few months.

We don't need AGI/ASI to impact the software industry; in my opinion we just need well targeted models that get better at a decent rate. At some point they either hit a wall or surpass people - time will tell BUT they are definitely targeting SWE's at this point.

[currymj]

i actually don’t think nontechnical people understand the result they want of code.

have you ever seen those experiments where they asked people to draw a picture of a bicycle, from memory? people’s pictures made no mechanical sense. often people’s understanding of software is like that — even more so because it’s abstract and many parts are invisible.

learning to clearly describe what software should do is a very artificial skill that at a certain point, shades into part of software engineering.

[throw234234234]

Think this is more true for more niche domains; but probably not for things like web/app development where the user can verify the output themselves. Its one of the reasons I'm more bearish on frontend/apps - because that's where the value is to most people and they understand it. That's the key and why it will disrupt code more than math - a non-math person doesn't actually know/want the input or output of advanced math (don't know what they don't know problem) so it remains more of a tool in that domain.

Those people with cross domain knowledge in an industry will continue to have value for some time able to contribute to domain discussions and execute better with the tech. As a result I've always thought the "engineering" part of software was more valuable than the CS/Leetcode part of the industry. As a lecturer many decades ago told me in a SE course - "you will know more about their business, in greater detail by the time you are finished, then they even do".

[throw83288]

I think what's missing is that the amount of training data to effectively RL usually decreases over time. AlphaGo needed some initial data on good games of Go to then recursively improve via RL. Fast forward a few years, and AlphaZero doesn't need any data to recursively improve.

This is what I mean by generalization skills. You need trillions of lines of code to RL a model into a good SWE right now, but as the models grow more capable you will probably need less and less. Eventually we may hit the point where a large corporations internal data in any department is enough to RL into competence, and then it frankly doesn't matter for any field once individual conglomerates can start the flywheel.

This isn't an absurdity. Man can "RL" itself into competence in a single semester of material, a laughably small amount of training data compared to an LLM.