Docker is just a container and not a sandbox. There are no controls in the kernel to prevent something from leaving the container unless you enable restrictions outside of docker such as SELinux. There is a lot of development around SELinux+Docker right now.
If you are connecting to X, it doesn't matter. You can attack all other X11 clients by design. This is why GNOME is pushing so heavily on xdg-app (for the sandbox) and the wayland-enabled GNOME shell (so apps can't snoop on each other).