[gortok]

In the comments here there are basically two schools of thought illustrated:

1. This is how MCP and LLMs work. This is how non-deterministic systems turn out. You wanted agentic AI. This is a natural outcome. What’s the problem?

2. We can design these systems to be useful and secure, but it will always be a game of whack-a-mole just like it is now, so what’s the problem?

What I’d like to see more of is a third school of thought:

3. How can anyone be so laissez-faire about folks using systems that are designed to be insecure? We should shut this down now, and let our sense guide our progress, instead of promises of VC-funded exits and promises of billions.

[c-linkage]

Most people don't want actual security. They want "good enough" security which provides the illusion of security. Because even when "good enough" security fails there are no consequences so "good enough" security is all we will ever get.

Data breaches are now as common as air disasters and school shootings, so no one except the victims cares anymore. And there aren't enough victims to matter in the financial sense.

It's always about risk, and business and the consumer have made a devil's bargain that the consumer will now take on more risk. LLMs will continue to be developed and used and MCP will worm its way into more and more things because software is accepted to be insecure by default and we all just live with it.

[simonw]

I propose a fourth school of thought:

4. A lot of obvious things we want to build with LLMs cannot be implemented in a secure way. Let's figure out the useful things that can be built securely and build those instead.