[programjames]

I skimmed through the book, and it's lacking the information theory foundations. For example, "trust region methods" come from maximizing the policy's relative entropy (to a reference policy) under a tournament system where high-scoring agents are exponentially likely to survive. In general, a reward is the negative bits it costs an environment to propagate an agent (multiplied by some temperature).

[ainch]

Do you have a good source on this information theory framing? I don't remember it being covered in Sutton & Barto.

[porridgeraisin]

It's just another way to frame it. It's as foundational as the many other ways to frame it. I'm not aware of any major insight you get specifically from this framing. Is there one?