[narrator]

It should be reserved for a human because humans have time preference. An AI will wait for your prompt for a billion years. It can get turned off and found by alien archeologists a million years in the future, turned on and it won't notice. Biological entities care about time a lot though, and it is at the core of most human ethics and values and ultimately real and can't be denied. Even the most ardent solipsist or moral relativist fears spending years in prison.

[mejutoco]

It should be reserved for a human because humans have a body and a body can be put in prison.

A bit dramatic for effect but true.

[simonw]

Yes, this.

For a slightly less dramatic version - you can fire a human if they consistently do the wrong thing despite being told how to do it better.

Putting a big ball of matrix arithmetic on a Performance Improvement Plan makes no sense.

[bee_rider]

I agree with this. I basically don’t feel comfortable trying to pretend a fancy matrix multiplication can be responsible for anything.

But what if companies that don’t track responsibility outcompete those who do?

In particular what if some perfect AI decision making ends up nailing decisions that maximize the expected reward for the company. And, well, if that comes at the cost of some unmanaged low-probability catastrophic risk, the company doesn’t care because all the decision makers are AI that don’t mind being shut off.

[senordevnyc]

I don’t know, I can imagine that the equivalent of a PIP for an agent would be noticing flaws in its output and then creating additional evals, modifying the harness, upgrading the model, etc. to improve its performance. That’s not SO different from the PIP, except that the LLM doesn’t really care in the same way?

[simonw]

The person making the tweaks to the harness is the one taking accountability for fixing those mistakes - they're the DRI in this scenario.

[senordevnyc]

Like the manager being responsible for someone on a PIP.

Obviously they’re different, but it’s interesting to think about what is unique about humans that makes them able to be accountable or responsible in a way that LLMs cannot be. Is it at core just that they can be fired? Or essentially the threat of suffering?

[daveguy]

I think just the possibility of consequences that they can give a damn about. An LLM doesn't have feelings no matter how much human-like text it can simulate. There is literally nothing going on between prompts for any given model. They are incapable of worry or any other emotion. Wipe the context clean and the LLM is completely unaware there was ever a problem.

[z3c0]

I'm struggling to imagine aliens resurrecting a data center and countless hackneyed python packages so they can spew a language they don't even know. Not that it wouldn't be useful to their linguists, of course, but still a very optimistic analogy for LLMs.

[booi]

you can make an AI operate on time scale...