[spinningslate]

Thanks, I enjoyed reading. As an electronic engineering student, I remember grappling with information theory in the abstract: it was a weather example very similar to yours that gave me the intuition I was missing.

An observation/suggestion. The intro is accessible to many people; that drops off a steep cliff when you hit the maths. Now, I'm not complaining about that: it's instructive and necessary to formalise things. Where I struggle is in reading the equations in my head when I don't know what words to use for the symbols. For example, that very first `X ~ p(x)`. I didn't know what to say for the tilde character, so couldn't verbalise the statement. I do know that $\in$ (the rounded 'E') means 'is a member of' so I could read the next statement. The problem gets even more confusing for a non-mathematician as the same symbol is used with different meaning in different branches of maths/science (e.g. $\Pi$).

I get that writing out every equation in English isn't feasible (or, at least, is asking a lot of the writer). But I wonder if there's middle way, e.g. through hyperlinking?

As I say: not a criticism and I don't have a good solution. Just an observation from a non-mathematician. Enjoyed the piece anyway.

[jessriedel]

"X ~ p(x)" means "X is a random variable drawn from the probability distribution p(x)" or maybe "X is drawn from p(x)" for short.

Are you sure it's a matter of knowing what to say (in your head) vs knowing the definition of the notation in the first place? I am pretty familiar with this notation, but I rarely verbalize it mentally. I can tell because I read and understand it quickly without problem, but on the rare occasion when I have to read it aloud I realize I'm not sure how I should pronounce it.

[spinningslate]

Thanks for the explanation.

Agree it's more "say in my head" than "speak out loud". But I still need to know what to say - internally or externally. Without knowing that ~ denotes "drawn from", all I can say is "X tilde p of x". That has no semantics; no intuition. Whereas knowing that $\in$ means "is a member of", I can read "x \in X" as "x is a member of X".

> but I rarely verbalize it mentally

Neither do I when I know something well. For example, I don't explicitly verbalise "is a member of" now, even internally. There's a shortcut hard-wired in that understands it without needing to pronounce it explicitly. In fact that short cut goes beyond the syntax: it goes straight to the intuition of "x represents any member of the set X". But I had to go through the process of saying it on the way to the shortcut.

[jessriedel]

OK, but if you know the formal definition, and you're not reading it out loud, why not just make something up? I actually don't know whether "is drawn from" is the "correct" way to pronounce the tilde. I think maybe other people say "is distributed as".

[windsignaling]

I don't think a piece on information theory should necessarily be "accessible to many people". It's a topic which is normally taught in grad school.

Something like X ~ p(x) would be seen all over the place in probability, stats, ML, and related courses such as info theory, detection and estimation, etc. Likely by the time someone is interested in info theory this notation would be permanently etched into their minds. So for this article it is very "audience appropriate".

> not a criticism and I don't have a good solution

Having a mental map of how different subjects fit together (without actually having to studying them in-depth) is a good start.

I've seen so many people crash and burn with machine learning because they were unaware that it depends on linear algebra, calculus, and probability.

With a mental map there is less "surprise" and it's more a matter of simply understanding that they didn't have the right dependencies.

[canjobear]

I read X ~ p(x) as "X is distributed as p of x"

[amelius]

What would X ~ p(y) mean?

[bllguo]

X is distributed as p(y) where p is a probability distribution parameterized by y