[ethanbond]

> Code hooked to a LLM that does durable actions in the real world should probably ask for human confirmation.

Right, and until this happens these systems are not HITL. The argument provided as recently as a few months ago that these systems are safe because humans will always be in the loop is now clearly dismissible.

[mlyle]

> Right, and until this happens these systems are not HITL.

You're drawing the system line strangely and making the choice about "in the loop" strangely.

A human decided to hook it up to a plugin with their Github credentials and to allow it to do actions without pre-approval. A human was still in the loop because the human then didn't like what it did and disconnected it. It only did a single action, rather than the kinds of scripting mistakes that I've seen that can do hundreds, but it still wasn't a very sane default for that plugin.

Is my cruise control HITL? It does not ask for my pre-approval before speeding up or slowing down.

[ethanbond]

Is a reasonable person surprised when their cruise control changes speed?

[mlyle]

Sometimes, yes. The radar my old Infiniti G35 used would sometimes get confused when facing into the sun in the late or early day and do bad things (in either direction: losing the car in front of it or decelerating unnecessarily). It was still HITL: I'd tap the brake and mitigate the bad thing it was doing.

HITL doesn't mean that a human never has to intervene or is never surprised by what the system does. It just means that a human initiates actions and can exercise genuine oversight and control.