Assuming that the server combines all the incoming streams into a single output stream, otherwise it still has to broadcast N-1 streams to each of the N users.
But why would you not combine them? In fact, I can easily see people being lazy and combining just on N sized stream to reduce server overhead especially if there is compression on top.