[jimmar]

People predict that in 50 years, no human will be driving a car, and people will be shocked that we let humans drive cars manually. Coding may be the same. So many vulnerabilities in code written by very competent programmers. Manually building large, complex systems without major bugs or security vulnerabilities seems to be a nearly impossible challenge.

[brightbeige]

And to consider AI agents are still mostly entirely limited to generating code in token-heavy programming languages designed to be written, tested and debugged by humans.

Here are two experimental exceptions:

https://github.com/vercel-labs/zerolang

https://github.com/sbhooley/ainativelang

[dmix]

Not just the languages but frontend/user interfaces as well. You can see the potential for the future when using Claude Design->Claude Code->Agents live testing in BrowserOS. It's all modeled on existing humans patterns of using Figma passing to devs then testing after the fact before starting the loop again, while a lot gets lost in translation in between the designs and the code.

We'll like have some standard AI-focused UI libraries that are harnessed into a design gen system where an AI can pull all the real levers, while also developing a large training data set around it.

[vb-8448]

I just wonder how many of those 1451 acknowledged findings were introduced by LLMs ...

[Oarch]

I reckon that in 50 years the very idea of code existing will be esoteric knowledge, a bit like binary. We simply won't care to think at that level of abstraction anymore.

[Gigachad]

In 50 years the world itself will be unrecognisable. The world could be a smouldering wreck by then.

[morpheos137]

there is little evidence for this prediction.

[scotty79]

What evidence would you expect to see if that was the case?

[sfink]

Some numbers, however shaky, that AI-written code is secure.

It could become that way, but thus far no evidence has been presented for it. The best we have right now is that you can spend $20 in tokens to write a patch and then $20K to find a vulnerability in it. First, that's not measuring the same thing. Second, it's not very impressive.

50 years is a long, long time, so I wouldn't bet against it. But I agree that we don't have evidence for it yet.

[scotty79]

What are the numbers on how secure is human written code? We should have something to compare AI numbers to.

It seems more likely to me that you could spend $20 to find a vulnerability in a piece of software that costed you $20k in human labor.

[sfink]

> What are the numbers on how secure is human written code? We should have something to compare AI numbers to.

That's kind of what the article is about? Mythos is finding lots of security bugs in lots of human-written code. They can now compute some sort of baseline estimate of security bugs per N lines of human-written code or whatever. (Restricted to security bugs that the AI is currently capable of finding, but whatever.) Even before Mythos et al, we can look at historical security bug rates. We do have numbers for estimating the security of human written code.

> It seems more likely to me that you could spend $20 to find a vulnerability in a piece of software that costed you $20k in human labor.

Ok, but that's not what is being discussed in this subthread? The topic is whether or not we have data suggesting that AI-written code is or can be secure, and thus whether insecure human code is fated to replaced with secure AI code. I claim we do not have that data. Therefore, we don't have evidence to think that for the sake of security we should replace all human code with AI code, vs whether AI code is worse for security and so we should replace AI code with human code (that presumably has been vetted with AI, since we do have evidence for its effectiveness.)

If I were to guess, I would probably think that today's AIs are trained solely on mountains of insecure human code and so will probably produce more of the same. Tomorrow's AIs will have the benefit of being trained on human and AI code that has had a large swathe of vulnerabilities purged from it, and so they'll have a much better chance at writing secure code, at least.

It depends a lot on whether the failure modes of AI code generation lend themselves to exploitation as security vulnerabilities. (And whether they will continue to do so.)

[scotty79]

> > It seems more likely to me that you could spend $20 to find a vulnerability in a piece of software that costed you $20k in human labor.

> The topic is whether or not we have data suggesting that AI-written code is or can be secure

I think my point is related because if the AI is great at finding vulnerabilities then it should be possible just tell AI to write the code and another AI to look for vulnerabilities and secure them. All for $20 + $20 instead of 20k.

Unless AI is somehow uncharacteristically weak in finding vulnerabilities in AI produced code. Which can probably be tested.

[cubefox]

The rapid progress in the last few years in this regard is pretty strong evidence in my opinion.

[morpheos137]

https://news.ycombinator.com/item?id=48225426

there is a difference between a stunt and a viable product. diverless cars and agi are the fusion of Silicon Valley.

[cubefox]

Unlike fusion, driverless cars are already a reality, there are just have a few kinks to work out. LLMs are also pretty close to AGI already. 50 years are more than enough to figure it out.

[sp527]

Oh there's plenty of evidence. Because a lot of these people have been committing to repos in public for over a decade. Wouldn't take much to show the world just how fallible human coders really are.

[cheesefck]

Musk has been predicting self driving cars next year for fifteen years. Fifty years ago, everyone was going to be flying supersonic all the time. Flying cars were just around the corner. Interplanetary travel. Everyone forgets the technology that fails.

This is the MoviePass era of language models

[Flere-Imsaho]

Actually I think with flying cars it's more of a problem with noise, regulation, risk, etc than a technological problem.

Supersonic again is a problem with noise and cost rather than technological.

Self driving is definitely a technological problem.

[3836293648]

I hope this will never be the case. As long as we have personal vehicles they should be personally controlled. Self driving cars is such a waste of everyone's money.

Cities should all have better public transport and out in the middle of nowhere you don't need self driving anyway. (And yes, personal cars should be entirely banned from cities)