The only thing that needs to live in the kernel is fast message interchange. And that could also be used as the base mechanism to move all(!) of the drivers out of the kernel.
Since Android 8 that the only in-kernel drivers are what the Android team considers legacy drivers, aka standard Linux kernel drivers pre-Project Treble.
Hence why modern Android drivers are called Binderized drivers.
I was just mulling this over yesterday with the story about USB tablet driver regression. Running drivers as user processes would essentially give the kernel a stable ABI. But then we'd see a proliferation of closed-source drivers - there wouldn't be motivation to upstream hardware support anymore. Is that a hidden goal behind this fast IPC thingy?