As balloon expands, in all directions there is same rate of expansion. You are not inside the balloon close to one side to observe the difference.