[bronson]

I'm tired of the framing in the media these days.

"Mythos will end the world!!"

"How?"

"By finding a bunch of wide open security holes that have existed for years."

Oookay. Is this a Mythos problem? Or a lazy/greedy/uncaring people problem?

[Arainach]

On the one hand, news coverage is overblown.

But scale and accessibility are absolutely a new class of problem.

In the 1960s you could pay thousands of people to watch hundreds of cameras and listen to hundreds of phone lines to monitor people, but the cost was so enormous that unless you were in East Germany or Moscow it wasn't a realistic threat model.

Now with computers we can cheaply have thousands of cameras with cheap storage that's retained forever and automatic image processing that means everyone is exposed to that kind of surveillance, which is a brand new problem.

[intrasight]

Storage is less cheap right now

I've often shared my prediction that future historians will study us all and that every living human will be the subject of someone's PhD thesis. I'm updating that prediction to be that those future someone's will be silicon based.

[Arainach]

Not by an order of magnitude.

Compared to "pay humans to watch hours of footage and take notes", $700 for a 16TB hard drive that will hold months of video footage is an absolute steal.

[afarah1]

Yes but also as always a people problem. People put the cameras and monitoring systems in place and operate them, to govern other people who ultimately yield to be governed, as the alternative is made too costly / dangerous by the governors.

[ben_w]

> as the alternative is made too costly / dangerous by the governors.

There's nothing government (corporate or political) has made "costly" or "dangerous" about not having them, society did that all by itself: people will actively pay more to have these things because they see the benefit more than the risk, video calls with friends and family, not a hacker being able to duplicate their keys from one photo.

There's 6 cameras on my desk right now, attached to internet-capable devices. Two phones, two laptops. I've got covers over all of them, which is easy, and sometimes mandatory e.g. when visiting the headquarters of certain big-tech firms.

I'd never buy a smart camera. Don't trust them not to spy on me. But I do have a Raspberry pi upstairs, with a NoIR camera module, and an AI-coded bit of webcam software. Might consider it for seeing what animal gets into the garden at night.

[ApolloFortyNine]

Heartbleed, one of the worst bugs in terms of exploitability and reach, was a bug that many engineers would be able to spot, if they were explicitly looking for it.

That's the risk of tools like Mythos/Fable/any LLM. While a human's eyes would glaze over what looks like a standard memcpy, an LLM with the right context might instantly realize the payload length was never actually verified.

And since Heartbleed existed for years, despite the full bug existing in pretty much one file, in one of the most important libraries out there, it's right to be afraid of what other obvious bugs exist and are just waiting to be found.

[ben_w]

> Oookay. Is this a Mythos problem? Or a lazy/greedy/uncaring people problem?

¿Porque no los dos?

All AI risk can be described with a narrative that ends in "some human were lazy and didn't care enough", it's just which humans and how much caring was enough.

[joshspankit]

It’s systematic

“6-12 mo to shaky profitability + ability to quickly iterate” is a business that has a good chance of surviving while “However long it takes to be fully secure” is a business that is not only rigid but needs massive up-front capital to get there and even then there’s no guarantee that the market fit is right

And after that is something we could call the “Pareto spiral”: if a company find market fit and builds an excellent product, competitors can survive at 80% of that quality. If the “100%” fails for any reason, the competitors become the new ceiling and now their competitors can survive at 80% of that (now 64%)

And only one round in, how secure could that 64% company be?

[cadamsdotcom]

Surely there's a competitive dynamic where if you really want to take down your competition you point your weaponized agent at their stuff and find some holes.

Seems given that the new security equilibrium baseline will settle much higher.

[joshspankit]

This has always been true to some extent: some have used weaponized security tools against competitors. It comes with legal risks (as it does with AI) but overall it doesn’t really improve the security baseline unless those holes are made public

[toomuchtodo]

People want simple answers to complex problems. When you find out most of society is held together with duct tape, promises, and trust, and then you have tool accelerant come along like GenAI, expectations violently meet reality. GenAI simply democratized the ability to evidence, inexpensively and at scale, in a wide variety of contexts, that "The Emperor has no clothes."

[afavour]

It’s still a problem we wouldn’t have without LLMs like Mythos existing. Yes, the solutions are different when you know the full context but “it’s just bad code” doesn’t mean there isn’t a problem.

[dgellow]

The AI labs are 100% responsible for that framing

[bdangubic]

If the prompt was “Who are 2026 Field Medal Winners” it is impressive regardless of the stupidity of people managing the website