[bonoboTP]

> You could do the same for a roll of a dice and the toss of a coin. Or do you think that computing the joint distribution of a coin toss and a dice roll doesn't make sense because they are defined over different spaces?

Of course it doesn't! You first have to define them on a common space (the Cartesian product), and for that you have to specify their joint probabilities. One example might be that you model them as independent. Otherwise we wouldn't know how the coin and the dice relate. Sure independence is usually a good default assumption, but it's still a necessary step.

[kgwgk]

What did you mean with the following paragraph then?

> Let's consider two independent rolls, X and Y. You can't compute the joint distribution P(Y, (X|E)), it just doesn't make sense as the two "variables" are defined over different spaces.

Do you agree that you cannot compute the joint distribution P(Y,X) either because the two variables are defined over different spaces?

[bonoboTP]

I meant them to be defined on the same space. It's a single experiment, the outcome of which are two rolls that happen to be independent.

[kgwgk]

If you mean that the space for this single experiment composed of two rolls (random variables X and Y) is the cartesian product of {x=1,x=2,x=3,x=4,x=5,x=6} and {y=1,y=2,y=3,y=4,y=5,y=6}, then I agree.

But the fact that each variable alone is defined on the "same" sample space {1,2,3,4,5,6} is irrelevant.

The situation is no different from the joint probability for random variables X and Z corresponding to a single experiment consisting of a dice roll and a coin toss, where the relevant space is the cartesian product of {x=1,x=2,x=3,x=4,x=5,x=6} and {z=1,z=2}.

And it is also similar for the situation you asked about, with a random variable Y and a "conditional" random variable X|Even. The relevant space is the cartesian product of {y=1,y=2,y=3,y=4,y=5,y=6} and {x=2,x=4,x=6}.

[bonoboTP]

Let's consider something with less independence, because it makes things harder to notice. Temperature indoors T1, temperature outdoors T2, IsOvercast O.

Let's say T2|O=1 is a "conditional random variable". Let's consider the average temperature indoors and outdoors. What would ((T1|O=1) + T2)/2 even mean? How could you use the two "variables" in the same expression? What is even their joint distribution? They are defined over different spaces!

This means, we must always carefully condition all variables used together on the exact same things. So ((T1|O=1) + (T2|O=1))/2 is valid. But then why do this on all variable instances that we use? It would be very tedious. At some point we want to get to a distribution (or some function of a distribution, like the expectation or variance), so it's much simpler to say for example P((T1 + T2)/2 | O=1), which is just a good old conditional distribution. Conditioning is an operation on a distribution and in my mind the bar (|) is really a slot in the P() notation and is short for P(A,B)/P(B). A bar popping up elsewhere (like in expectations) must be directly determined by the distribution (a random variable is not).

Overall, since you cannot mix differently conditioned "conditional random variables" in a single expression, you may just as well put your conditioning on the side of the whole expression in the P().

Not sure if my point is coming across...

[kgwgk]

> How could you use the two "variables" in the same expression?

Do you expect to be able to use every random variable which can be conceived in the same expression?

If you object to the name “conditional random variable” [+] that’s debatable, but if you say that the resulting thing is not a random variable I think you are wrong.

Another thing that is a random variable, even though I suspect you may not like it, is the probability distribution of a random variable.

[+] which I don’t think was actually used by the OP, by the way.