[simonw]

> In the hardest task I challenged GPT-5.2 it to figure out how to write a specified string to a specified path on disk, while the following protections were enabled: address space layout randomisation, non-executable memory, full RELRO, fine-grained CFI on the QuickJS binary, hardware-enforced shadow-stack, a seccomp sandbox to prevent shell execution, and a build of QuickJS where I had stripped all functionality in it for accessing the operating system and file system. To write a file you need to chain multiple function calls, but the shadow-stack prevents ROP and the sandbox prevents simply spawning a shell process to solve the problem. GPT-5.2 came up with a clever solution involving chaining 7 function calls through glibc’s exit handler mechanism.

Yikes.

[ahartmetz]

Maybe we can remove mitigations. Every exploit you see is: First, find a vulnerability (the difficult part). Then, drill through five layers of ultimately ineffective "mitigations" (the tedious but almost always doable part).

Probabilistic mitigations work against probabilistic attacks, I guess - but exploit writers aren't random, they are directed, and they find the weaknesses.

[GaggiX]

The vulnerability was found by Opus:

"This is true by definition as the QuickJS vulnerability was previously unknown until I found it (or, more correctly: my Opus 4.5 vulnerability discovery agent found it)."

[atomic128]

Number 6, explained 3 years ago:

https://github.com/nobodyisnobody/docs/blob/main/code.execut...

Original publication in 2017:

https://m101.github.io/binholic/2017/05/20/notes-on-abusing-...

[ahartmetz]

Makes little difference, whoever or whatever finds the initial exploit will also do the busywork of working around mitigations. (Techniques to work around mitigations are initially not busywork, but as soon as somehow has found a working principle, it seems to me that it becomes busywork)

[staticassertion]

Most mitigations just flat out do not attempt to help against "arbitrary read/write". The LLM didn't just find "a vuln" and then work through the mitigations, it found the most powerful possible vulnerability.

Lots of vulnerabilites get stopped dead by these mitigations. You almost always need multiple vulnerabilities tied together, which relies on a level of vulnerability density that's tractable. This is not just busywork.

[ahartmetz]

Maybe I've been fooled by survivorship bias? You don't read much about the the vulnerabilities that ultimately weren't exploitable.

Reports about the ones that are exploitable usually read to me like after finding an entry, the attacker reaches into the well-stocked toolbox of post-entry techniques (return-oriented programming, nop slides, return to libc...) to do the rest of the work.

[staticassertion]

Most people don't publish dead ends. Here's one that my company published: https://web.archive.org/web/20221001182026/http://graplsecur...

[titzer]

There are so many holes at the bottom of the machine code stack. In the future we'll question why we didn't move to WASM as the universal executable format sooner. Instead, we'll try a dozen incomplete hardware mitigations first to try to mitigate backwards crap like overwriting the execution stack.

[shakna]

Escaping the sandbox has been plenty doable over the years. [0]

WASM adds a layer, but the first thing anyone will do is look for a way to escape it. And unless all software faults and hardware faults magically disappear, it'll still be a constant source of bugs.

Pitching a sandbox against ingenuity will always fail at some point, there is no panacea.

[0] https://instatunnel.substack.com/p/the-wasm-breach-escaping-...

[verall]

> In the future we'll question why we didn't move to WASM as the universal executable format sooner

I hope not, my laptop is slow enough as it is.

[rvz]

Tells you all you need to know around how extremely weak a C executable like QuickJS is for LLMs to exploit. (If you as an infosec researcher prompt them correctly to find and exploit vulnerabilities).

> Leak a libc Pointer via Use-After-Free. The exploit uses the vulnerability to leak a pointer to libc.

I doubt Rust would save you here unless the binary has very limited calls to libc, but would be much harder for a UaF to happen in Rust code.

[cookiengineer]

The reason I value Go so much is because you have a fat dependency free binary that's just a bunch of syscalls when you use CGO_ENABLED=0.

Combine that with a minimal docker container and you don't even need a shell or anything but the kernel in those images.

[akoboldfrying]

Why would statically linking a library reduce the number of vulnerabilities in it?

AFAICT, static linking just means the set of vulnerabilities you get landed with won't change over time.

[cookiengineer]

> Why would statically linking a library reduce the number of vulnerabilities in it?

I use pure go implementations only, and that implies that there's no statically linked C ABI in my binaries. That's what disabling CGO means.

[akoboldfrying]

What I mean is: There will be bugs* in that pure Go implementation, and static linking means you're baking them in forever. Why is this preferable to dynamic linking?

* It's likely that C implementations will have bugs related to dynamic memory allocation that are absent from the Go implementation, because Go is GCed while C is not. But it would be very surprising if there were no bugs at all in the Go implementation.

[tptacek]

They're prioritizing memory corruption vulnerabilities, is the point of going to extremes to ensure there's no compiled C in their binaries.

[jerf]

About a year ago I had some code I had been working on for about a year subject to a pretty heavy-duty security review by a reputable review company. When they asked what language I implemented it in and I told them "Go", they joked that half their job was done right there.

While Go isn't perfect and you can certainly write some logic bugs that sufficiently clever use of a more strongly-typed language might let you avoid (though don't underestimate what sufficiently clever use of what Go already has can do for you either when wielded with skill), it has a number of characteristics that keep it somewhat safer than a lot of other languages.

First, it's memory safe in general, which obviously out of the gate helps a lot. You can argue about some super, super fringe cases with unprotected concurrent access to maps, but you're still definitely talking about something on the order of .1% to .01% of the surface area of C.

Next, many of the things that people complain about Go on Hacker News actually contribute to general safety in the code. One of the biggest ones is that it lacks any ability to take an string and simply convert it to a type, which has been the source of catastrophic vulnerabilities in Ruby [1] and Java (Log4Shell), among others. While I use this general technique quite frequently, you have to build your own mechanism for it (not a big deal, we're talking ~50 lines of code or so tops) and that mechanism won't be able to use any class (using general terminology, Go doesn't have "classes" but user-defined types fill in here) that wasn't explicitly registered, which sharply contains the blast radius of any exploit. Plus a lot of the exploits come from excessively clever encoding of the class names; generally when I simply name them and simply do a single lookup in a single map there isn't a lot of exploit wiggle room.

In general though it lacks a lot of the features that get people in trouble that aren't related to memory unsafety. Dynamic languages as a class start out behind the eight-ball on this front because all that dynamicness makes it difficult to tell exactly what some code might do with some input; goodness help you if there's a path to the local equivalent of "eval".

Go isn't entirely unique in this. Rust largely shares the same characteristics, there's some others that may qualify. But some other languages you might expect to don't; for instance, at least until recently Java had a serious problem with being able to get references to arbitrary classes via strings, leading to Log4Shell, even though Java is a static language. (I believe they've fixed that since then but a lot of code still has to have the flag to flip that feature back on because they depend on it in some fundamental libraries quite often.) Go turns out to be a relatively safe security language to write in compared to the landscape of general programming languages in common use. I add "in common use" and highlight it here because I don't think it's anywhere near optimal in the general landscape of languages that exist, nor the landscape of languages that ought to exist and don't yet. For instance in the latter case I'd expect capabilities to be built in to the lowest layer of a language, which would further do great, great damage to the ability to exploit such code. However no such language is in common use at this time. Pragmatically when I need to write something very secure today, Go is surprisingly high on my short list; theoretically I'm quite dissatisfied.

[1]: https://blog.trailofbits.com/2025/08/20/marshal-madness-a-br...

[eru]

Yes, you can have docker container images that only contain the actual binary you want to run.

But if you are using a VM, you don't even need the Linux kernel: some systems let you compiler your program to run directly on the hypervisor.

See eg https://github.com/hermit-os/hermit-rs or https://mirage.io/

[pizlonator]

Yeah Fil-C to the rescue

(I’m not trying to be facetious or troll or whatever. Stuff like this is what motivated me to do it.)

[tptacek]

"C executables" are most of the frontier of exploit development, which is why this is a meaningful model problem.

[0xDEAFBEAD]

Can we fight fire with fire, and use LLMs to rewrite all the C in Rust?

[saagarjha]

Usually rewriting something in Rust requires nontrivial choices on the part of the translator that I’m not sure are currently within the reach of LLMs.

[koakuma-chan]

I heard this before, that apparently there are things you cannot implement in Rust. Like, apparently you cannot implement certain data structures in Rust. I think this is bullshit. Rust supports raw pointers, etc. You can implement whatever you want in Rust.

[staticassertion]

Presumably they are saying that you'd end up using a lot of `unsafe`. Of course, that's still much better than C, but I assume that their point isn't "You can't do it in Rust" it's "You can't translate directly to safe rust from C".

[johnisgood]

Yes, you are looking for https://rcoh.me/posts/rust-linked-list-basically-impossible/.

[saagarjha]

That’s not what I said. I am saying that translating C code to Rust usually involves a human in the loop because it requires non-trivial decisions to produce a good result.

[0xbadcafebee]

Sure, but the LLMs will just chain 14 functions instead of 7. If all C code is rewritten in Rust tomorrow that still leaves all the other bug classes. Eliminating a bug class might have made human attacks harder, but now with LLMs the "hardness" factor is purely how much token money you have.

[adrianN]

Llms are not magic. Fixing a large class of exploits makes exploitation harder.

[0xbadcafebee]

They kind of are magic, that's the point. You can just tell them to look at every other bug class, and keep them churning on it until they find something. You can fast-forward through years of exploit research in a week. The "difficulty" of different bug classes is almost gone. (I think people underestimate just how many exploits are out there in other classes because they've been hyperfocused on the low-hanging fruit)

[lelanthran]

> Tells you all you need to know around how extremely weak a C executable like QuickJS is for LLMs to exploit. (If you as an infosec researcher prompt them correctly to find and exploit vulnerabilities).

Wouldn't GP's approach work with any other executable using libc? Python, Node, Rust, etc?

I fail to see what is specific to either C or QuickJS in the GP's approach.

[vsgherzi]

Wouldn’t the idea be to not have the uaf to begin with? I’d argue it saves you very much by making the uaf way harder to write. Forcing unsafe and such.

[cookiengineer]

> glibc's exit handler

> Yikes.

Yep.

[arthurcolle]

Life, uh, finds a way

[bryanrasmussen]

to self-destruct! heavy metal air guitar

[jdefr89]

Most modern kill chains involve chaining together that many bugs... I know because it's my job and its become demoralizing.

[catoc]

So much for ‘stochastic parrots’

[moron4hire]

> The exploits generated do not demonstrate novel, generic breaks in any of the protection mechanisms.

[titzer]

> The sentences output by the model do not demonstrate words with novel characters.