I was at one point trying to get GPU passthrough working with LXC and I could never get programs to actually run. Ubuntu 16.04, CUDA 7.5 or maybe 8.0, and a cheap GTX 730.
Maybe I'll have to try again. I was also trying this so that I could run my desktop environment in a container too.
There is no passthrough in LXC, because it is containerization, not virtualization. You are running the same kernel and kernel modules for both systems.
Yes, I had installed the Nvidia drivers on both the host system and the container. The CUDA programs run on the host, but didn't inside the container, even after creating the proper device nodes in /dev in the container.
My guess would be, that the user space driver uses additional mechanism to communicate with the kernel module. I don't thing that it hauls large data buffers via device files; I would try strace to see, where it fails.