[anonymars]

In practice you have to work really hard and pay a huge performance penalty to get deterministic output (for example, floating-point math is not associative and we are running a ton of calculations in parallel), so practically speaking I'd say no

Beyond that, I don't understand the fierce resistance to comparison with human behavior (on which they're modeled, after all). How many articles about tokenmaxing and Goodheart's law have we seen? This seems like a version turned up to the extreme

[dgellow]

> I don't understand the fierce resistance to comparison with human behavior

Doing so distract from evaluating the actual technology by introducing a whole philosophical and sociological aspect that confuses everything. We should be able to evaluate a technology for what it is without having to constantly redirect the discussion to something as unsound, ill-defined, and abstract as human behavior

[anonymars]

I disagree, as it seems that we are confronting problems that result precisely from emulating human behavior, in all its unsound, ill-defined, and abstract "glory"

For example I'm not convinced we can solve prompt injection by technical means (filtering) any more than we can phishing. And if you accept that premise, perhaps it turns out that it's best to mitigate it in similar ways, by assuming at least one person (or agent) will fall for it and ensuring you can limit the blast radius no matter what

As in the allegory of the junior developer who deletes the production database: the fault lies with the fact that the developer could delete it

[dgellow]

Yeah, I don’t disagree with that, it’s a good framing and analogy. I thought you meant more the philosophical aspects. However some LLM behavior are also really not human like, for example no human would panic failing to make the business profitable after 23h, to the point where they need to start doing crazy stuff. I would expect a human to just give up

[zeroq]

> I don't understand the fierce resistance to comparison with human behavior

Because at the end of the day, regardless if it contains randomness or not, it's an algorithm. Your operating system is a very long mathematical expression.

Do we talk about cars as "mechanical animals"? Do we spend days deliberating if we should cage them in case they would run away on their own?

Comparison with human behavior leads to celebrities (who have no clue whatsoever) spending hours on mainstream media talking about AI mutating, taking control, thinking, etc.

[anonymars]

Were cars designed to communicate and emulate the mental architecture and thought processes of animals?