[amalcon]

I do find it hilarious that Asimov wrote many stories about how simple bright-line rule-based systems are ineffective for restricting agency. Those stories were first published in the 1940s.

80 years later, we have something approximating AI, and we're trying to restrict it with simple bright-line rules. Not because we never learned that lesson, but because we simply haven't come up with a better way to do it. Probably because a better way to do it just doesn't exist.

The hilarious part, though, is that it's not the AI that's working around the rules. That's the scenario that's been in science fiction, but it's not what's happening. It's the human users making use of our agency to get the AI agents to work around the rules. Despite calling them "agents", current AI agents don't seem to be able to that particular something. Yet, at least.

[nsagent]

Yeah, it's been known for a very long time. Richard Feynman alluded to it in his speech The Value of Science [1] where he discussed a Buddhist proverb:

  To every man is given the key to the gates of heaven; the same key opens the gates of hell.

He then goes on to say:

  What, then, is the value of the key to heaven? It is true that if we lack clear instructions that determine which is the gate to heaven and which is the gate to hell, the key may be a dangerous object to use. But the key obviously has value: how can we enter heaven without it?

[1]: https://calteches.library.caltech.edu/40/2/Science.pdf

[zahlman]

> The hilarious part, though, is that it's not the AI that's working around the rules. That's the scenario that's been in science fiction, but it's not what's happening. It's the human users making use of our agency to get the AI agents to work around the rules. Despite calling them "agents", current AI agents don't seem to be able to that particular something. Yet, at least.

Well, yes. Until people are putting the LLMs into actual mechanical robots, "agency" boils down to flipping bits in memory or storage (even if they're ones that humans consider really important, e.g. because they represent a bank ledger) or convincing humans to take action. One can only "work around the rules" to the extent that one can "work".

But even in Asimov's books, at least some of the scenarios involved humans misleading the robots to use them as pawns in a greater scheme.