[mlyle]

Humans misuse systems all the time and are surprised, even in safety critical regimes.

Sometimes the system design is insufficient (I implied above the plugin could be a little better).

I hate blaming the user instead of the system, but sometimes the user deserves the blame, too. Sometimes it really just is pilot error.

[ethanbond]

Assign blame wherever you want, the fact of the matter is this is not what most people mean when they say “human in the loop.” The “AI will always have HITL” argument was always weak, but now plainly disproven.

The logged behavior would surprise many totally sensible people, as you’re seeing in this comment thread.

What exactly was the user error? Are we to believe that if you authenticate a plug-in into your session you are okaying it to do any of its supported operations, even at wildly unexpected times, and this is considered “in the loop?”

[mlyle]

> Are we to believe that if you authenticate a plug-in into your session you are okaying it to do any of its supported operations, even at wildly unexpected times, and this is considered “in the loop?”

Here, someone chose to run code and give it credentials. The code was designed, among other things, to let ChatGPT open issues. They were surprised when the code opened an issue on behalf of ChatGPT using the user's credential.

When you run code on a computer designed to do X and give it credentials sufficient to do X, you may expect that X may occur. This isn't really an AI issue.

Code hooked to a LLM that does durable actions in the real world should probably ask for human confirmation. It's probably a good practice of plugin developers to have some distinction similar to GET vs. POST.

Most code that would automatically open issues on GitHub should probably ask for human confirmation. There's some good use cases that shouldn't, including some with LLMs involved -- but asking is a sane default.

I remember being surprised when I ran a program and it sent a few hundred emails once.

[ethanbond]

> Code hooked to a LLM that does durable actions in the real world should probably ask for human confirmation.

Right, and until this happens these systems are not HITL. The argument provided as recently as a few months ago that these systems are safe because humans will always be in the loop is now clearly dismissible.

[mlyle]

> Right, and until this happens these systems are not HITL.

You're drawing the system line strangely and making the choice about "in the loop" strangely.

A human decided to hook it up to a plugin with their Github credentials and to allow it to do actions without pre-approval. A human was still in the loop because the human then didn't like what it did and disconnected it. It only did a single action, rather than the kinds of scripting mistakes that I've seen that can do hundreds, but it still wasn't a very sane default for that plugin.

Is my cruise control HITL? It does not ask for my pre-approval before speeding up or slowing down.

[ethanbond]

Is a reasonable person surprised when their cruise control changes speed?

[mlyle]

Sometimes, yes. The radar my old Infiniti G35 used would sometimes get confused when facing into the sun in the late or early day and do bad things (in either direction: losing the car in front of it or decelerating unnecessarily). It was still HITL: I'd tap the brake and mitigate the bad thing it was doing.

HITL doesn't mean that a human never has to intervene or is never surprised by what the system does. It just means that a human initiates actions and can exercise genuine oversight and control.