[golwengaud]

I found https://www.lesswrong.com/posts/no5jDTut5Byjqb4j5/six-and-a-... very helpful for getting intuition for what the K-L divergence is and why it's useful. The six intuitions:

    1. Expected surprise
    2. Hypothesis testing
    3. MLEs
    4. Suboptimal coding
    5a. Gambling games -- beating the house
    5b. Gambling games -- gaming the lottery
    6. Bregman divergence