[rgoulter]

Seems to me LLMs have changed some things. I'm not sure how it's best put, but it used to be:

- Seeing code (or a blogpost or whatever) was a result from effort where thought had gone into it. The writer paid effort so the reader didn't have to.

- There'd be some level of attachment to what you've put effort into.

With LLMs, that's undermined: it's easy to produce thoughtless imitations. Code or comments where thought didn't go into it. So, seeing some result isn't an indication of skill, but also not even an indication thought went into it.

I guess there's still something lost if someone isn't going to share code they've put thought into. -- But on the other hand, if it's just for me & I don't have to share it with a wider audience, getting LLMs to write out code isn't so expensive.. so code itself isn't necessarily something to value so much.

[irdc]

But LLMs don’t seem particularly good at inventing new ways to code (or write, or…). It’s literally all derivative. So what happens in 10 years? Are we headed for a great stagnation?

[rgoulter]

> But LLMs don’t seem particularly good at inventing new ways to code (or write, or…). It’s literally all derivative.

I think the key part is how much thought goes into something.

Optimistically, LLMs are good at taking unstructured input, and (probably) producing the intended output from that. -- This allows for an interesting new way of coding: a set of instructions don't need to be as rigorous as a shell script, but can be natural language.

That part surely extends creativity. An LLM will be familiar with domain ideas I'm not, even if an LLM is completely disinterested in doing things.

Pessimistically, I think it's still not clear what the right way of interacting online with all of this is (other than clear expectations of "no AI")... in some sense LLM output is worthless to share, in the sense that I'm just as capable of asking the LLM to output something as anyone else is.

[mswphd]

Looking at LLMs applications to math might be instructive. A year ago when they had some preliminary claims/results, people would hypothesize they had the answer implicit in their training data (and so were being "better search", but fundamentally doing derivative work). This may have been substantiated sometimes, I forget.

Recently the tune has changed somewhat, say with LLM's approaches to Erdos problems (and in particular the unit distance problem. The LLM solution here spurred progress on another large problem, namely https://arxiv.org/abs/2605.28781 ). There have been no claims that the LLMs work on the unit distance problem was derivative, and I've seen mathematicians claim it would have been accepted to a top journal (say Annals).

In spite of this, the capabilities of LLMs within mathematics are still limited. LLMs seem decent at

1. "constructions", e.g. where you claim \exists object with certain properties. It can help if the verification that the object has these properties is efficiently computable, but I don't believe this sort of verification was used for the unit distance problem.

There are other areas of math that LLMs so far are less adapted to, for example

2. impossibility results, or showing \lnot \exists object with certain properties, or

3. "abstraction building". Often in math results become much easier to obtain if you have "the right definition". Grothendiek was famous for this, as is e.g. Scholze currently.

These claims are based off of current public results via LLMs. It's possible capabilities will develop further. But also, in hindsight, it is natural that LLMs would be better at the thing they ended up being good at.

I'm unsure if there is a way of extracting from this insights to programming/writing. Plausibly, you could see LLM's developments of PoC exploits as similar to (1) but for computer science. It is a concrete "construction" that is efficiently verifiable. (2) would suggest trivial observations that it would be hard for ah LLM to show that a program does not have vulnerabilities. I'm not sure if there are less trivial observations. Finally, (3) might be what you're bemoaning. In simple language, it would currently be surprising if LLMs could create useful, novel, design patterns/abstractions.

[uncircle]

Let LLMs ingest its own output, everything past 2022 will be increasingly hallucinatory self-regurgitation.

[multjoy]

That’s because they cannot invent anything. They’re reductive, not creative.

[dzhiurgis]

It’s like arguing that nobody is going to invent new ways to ride horses in the age to automobile.

[irdc]

If the way humanity advances were via new ways to ride horses, then yes.

[asibahi]

You made me curious. Has anyone invented new ways to ride horses in the age of the automobile?

[VladVladikoff]

Best I could find: https://www.science.org/doi/10.1126/science.1174605

There was a relatively big shift in riding style right around the same time of the first mass production of vehicles.

[f6v]

I don't know... I've been writing code for good twenty years (15 professionally).

First, I think it's the best time to write software since so much boring stuff can be automated. I can put my thoughts into what I'm trying to achieve instead of how. To put it otherwise, I think about big picture much more than about mundane details like dealing with particularities of a programming language.

Second, most people were using SO to solve just about any issue they had. The number of developers producing truly original code was minimal even 10 years ago.