Hacker News new | ask | show | jobs
by eru 2 hours ago
> Opus 4 class models are terrifying at infosec. They tie their shoelaces together on other things, but don't fuck with them on that. It's a savant thing.

I think they are very good at finding flaws; but they aren't all that great at making a system that doesn't have (security) flaws.
2 comments
tptacek 2 hours ago
What makes you say that? I think they're better than replacement-level developers at making secure systems (I spent 20 years looking for vulnerabilities in human-written code as a full-time job).
link
eru 1 hour ago
See https://news.ycombinator.com/item?id=48640533 for some further elaboration.

These models are definitely a lot better than your run of the mill human developer at finding security flaws in existing systems. I'm agnostic at how good they are at actually making a secure system. Probably better, too, for two reasons:

- humans are really terrible

- the model probably has an easier time picking up special purpose tools you can use to write proven secure systems

I don't think Mythos can write secure C code, either. Practically no one can. (At least not directly. See how seL4 is officially written in C; but they didn't just set out to carefully write secure C code directly; C just happens to be an intermediate language they use.)

link
sscaryterry 2 hours ago
Agreed. In the right hands, they can perform magic.
link
reinitctxoffset 2 hours ago
You are not wrong, but there's an asdymetry here: run adversarial self play and low-pass filter.
link
eru 2 hours ago
Mostly right. However there's an extra assumption I didn't explicitly state:

Almost all existing real world software is full of holes and security flaws. Mythos is better than humans at uncovering many of them; especially because its time is a lot cheaper than that of the top tier human experts (and even of mid-and low-tier human experts).

Especially when these systems are written in notoriously unreliably languages like C.

I don't think Mythos is especially good at writing systems that are free of security problems. Essentially the only way we know is by proving your software correct.

In principle, you can even prove C correct, but in practice you'll want to write your system from the ground up to be proven correct instead of adding that property after the fact; and for that you'll most likely also want to pick a language that supports this better.

See https://en.wikipedia.org/wiki/SeL4 for a noteworthy example.

link