[MattSayar]

> The loudest reaction to Mythos Preview from other security leaders has been about speed - scan faster, patch faster, compress the response cycle. More than one team we have spoken with is now operating under a two-hour SLA from CVE release to patch in production [...] If regression testing takes a day, you cannot get to a two-hour SLA without skipping it, and the bugs you ship when you skip regression testing tend to be worse than the bugs you were trying to patch.

Over time, I wonder if these models will be able to generate more secure code by default by doing this kind of exploitability testing before ever merging their code.

[krupan]

I don't know, but it always seems weird to me when people notice AI isn't performing super well and then they conclude that the solution to problem is to try using more AI

[tskj]

Yeah why not? That's how I work. If I don't review my work, it's way worse than if I do review it and revise and iterate. I don't see why AI should be different: in fact it very clearly seems to be the case that is isn't.

[krupan]

I mean, I was sold something different. Something super human, vastly more intelligent, world changing. The reality is not that. Am I allowed to be disappointed and discouraged?

[tskj]

Because you can have it review itself and iterate on its own work before showing you. If you insist on reviewing its one-shot output you'll be disappointed, but if you consider its internal work private and only consider its final output, it's different.

Also we're still in the middle of the transformation, clearly the AI we'll have in 5 years will be radically different and better (by some definition of better) than what we see today. It's kind of weird that you'd be disappointed that the world will only be totally transformed in ten years, and not five.

[krupan]

"clearly the AI we'll have in 5 years will be radically different and better"

Based on past performance that's not clear at all. Remember Elon predicting full self driving by 2017. It's almost 10 years past that predicted date and it's still not quite there. 5 years is nothing in tech. It takes 5 years to get a slightly improved chip designed and manufactured. It's been 3.5 years since ChatGPT was released and the LLMs of today are not radically different from that, and no radical changes have been teased. We're still in the throw-more-hardware-at-it phase. We could be here a while.

[HDBaseT]

It has changed the world in major ways, although its not entirely visible because we've become numb to the idea of AI and AI being in everything.

It hasn't changed the way we sleep, wake up, eat, walk and talk so its not "life changing" or "world changing" in the sense a meteorite hit us, but each day thousands of mini meteorites are hitting Earth and we're becoming normalized to it one step at a time.

You are allowed to be disappointed and discouraged! For all the good tech that has come out of the AI revolution, most of it is ignored or shelved for things that can squeeze more and more money out of us and make our lifes worse, not better. Despite there being real potential to generate nice code, assist with biomedical research, self-driving cars, etc.

[krupan]

Which is it? Major changes or a bunch of small changes. I'm well aware of the small changes. I worked for an autonomous drone company back in 2008. It was really cool! In 2020 I started working for an autonomous car company. Again, amazing! None of it was a quick step function improvement. It was a lot of hard work. None of it was quite superhumanly smart either. LLMs are impressive pattern completion machines but they kinda suck at producing anything truly novel. Plus they are compulsive liars about that, lol!

[germandiago]

Reminds me of people adding more intervention and bureaucracy bc the last one did not do well, so we need more of it.

The problem is never the results of it. It is that we did not do well enough.

[edu]

Or they don’t, and they* sell access to Mythos and successors through their services company or network of partners and charge a premium.

* they, I mean all foundation models providers, as OpenAI seems to go in the same direction