[jonfw]

I used to work with a guy who would always say "if you're looking for trouble, you are going to find it"

When I hear that "we found X bugs using some new tool", where the standard for bugs is low and doesn't neccessarily require user impact in realistic scenarios, I think to myself- duh! You went looking for bugs, of course you found them.

For a sufficiently complicated product, in my experience, you don't have to look far.

[tedbradley]

Sure, but the bugs were found in an automated process. They just let an LLM scan. That's very impressive finding 100s of needed code changes. And it's even better if those needed code changes are bugs / vulnerabilities. The part no one is talking about comes from the bill. I'm sure Anthropic let Mythos analyze possibly for US$10,000s in tokens. A similar phenomenon happened back when an LLM scored well on some math olympiad competition. Yeah, it got all the answers right, but it was a frontier model running for 8 hours straight. That'll hurt the budget quite a bit. We're likely not at a stage where big corporate systems can just throw Mythos at it willy nilly for a complete analysis unless they have a ton of money.

[apexalpha]

Well it helps if 'looking for bugs' doesn't cost $300 per hour per set of eyes.

[metalliqaz]

how much does it cost? my understanding of Mythos is that it runs a lot to find issues

[ls612]

The things I’ve read from various open source orgs with access to it is that Anthropic is giving them unmetered access for now as part of Glasswing. I’d bet that the corporate partners have to pay though.

[tialaramex]

> if you're looking for trouble, you are going to find it

That's the "'No Way to Prevent This,' Says Only Nation Where This Regularly Happens" of unsafe languages.

There are huge swathes of problems we know how to categorically prevent, but some people won't do it because they're more comfortable believing it was never preventable than accepting any culpability for not preventing it previously.