[merlindru]

this is basically trying to enforce security-by-obscurity, which is a terrible idea all around. it's just a model. the security issues still exist and are exploitable.

and after staking the economy on AI, you can't really put a cap on intelligence. if models are not allowed to be better than Opus 4.8, then the whole investment structure is about to unravel.

why invest billions and billions into AI if returns are artificially capped?

[softwaredoug]

Especially as inference gets cheaper, open models proliferate, and it all just becomes ubiquitous and commoditized.

You can’t keep this genie in its bottle for long.

[uejfiweun]

Wow, it's starting to seem like the choice is essentially between an intelligence cap that pops the bubble, and an increasingly chaotic and unpredictable cybersecurity environment with major hacks and exploits left and right.

[merlindru]

But those hacks and exploits have always existed. Just had to have the right people to find them / be sufficiently motivated.

The same models that can find these exploits can also help fix them, thus everyone will be better off.

Relying on the fact that nobody has found a security issue with a piece of software yet is not a great way to ensure safety