[nchagnet]

Hi, author here! Thanks for the feedback, as I mentioned this is also to clarify things for myself so this helps a lot.

Regarding your points:

- I'm not sure I get your meaning here. My understanding is that for a random variable X, thr surprise is defined at the outcome level I(x) = - log p(x) while the entropy is essentially just the average value - sum_x p(x) log(p(x)). So to me it does look like entropy is expected surprise no? I do agree though that by being _expected_ surprise, entropy is itself a measure of surprise.

- I very much agree with that which is why I used _excess_ surprise (maybe relative is a better choice, but the intent is the same).

- That one I'm also confused about. It gets back to my first point: to me surprise (or information) is always defined at the outcome level first, so taking a moment is not tautological, it's meaningful, no?