You don't need to mount the /dev entries into the container at all. The experimental support creates them automatically for you when you are using GPU resources. Perhaps it's device nodes, not the libraries that required privileges?
Aaah that is interesting. Let me dive into this later today and test my charts without that. It would actually make my life way easier for charting.
I got that from a very early stage work and never questioned it again (the /dev stuff). Thanks for pointing that out.
OK I gave it a try and you are absolutely right. For the nvidia-smi, I could run it the /dev/nvidia0, which is cool.
I was also able to run it unprivileged. I guess my mistake was to believe the example from the docs and not test without.
Thanks for sharing that, I'll update my charts and the post accordingly.