If you know what the devices you are likely to plug in, you could just modprobe them all before disabling it.