[newsomix9xl]

Real biological operant behavior isn't exactly trial and error learning.

Many factors shape and guide initial responses.

What I've noticed in some descriptions of models is the use of optimization for reinforcement to shape responses. In real organisms behavior may be controlled by short or long term outcomes, and may oscillate between this "optimization" based on schedules. This produces variability in the trials which can adjust behavior. Are we seeing these reinforcement models do this?

[herodoturtle]

I found this comment/question deeply intriguing.

I’m no expert at this and was wondering what you meant by the following:

> In real organisms behavior may be controlled by short or long term outcomes, and may oscillate between this "optimization" based on schedules

Could you perhaps provide an example that would help me understand what you mean?

Thanks for the insightful comment either way.

[newsomix9xl]

In humans this is often called impulsivity, the preference for smaller sooner outcomes. This is often seen in children and animals and in some adult human behavior.

An impulsive choice is not optimal. You can buy a cheaper pack of gum at Costco in a week or get one for three times the cost right now.

[ainch]

There is a field of hierarchical RL in which the optimisation occurs over a range of time scales/abstraction. But I'm not aware of much practical success for these approaches so far.