Qualcomm has no obligation to GPL their kernel drivers. Nor does nvidia, or any other KLM maker. This has never demanded that kernel drivers be open sourced. That's why it's impossible to make a runnable AOSP image for many devices.
Vendors still make generally minor changes to the kernel, but these are not unique or special or some competitive edge. They've just nuisance necessities.
It's way more grey than that. Nvidia claims that their driver doesn't need to be opened because since it's A) from a codebase that existed prior to the Linux port, and B) doesn't actually import any Linux code or symbols (that's left to a dual licensed shim layer), that in a legal sense their driver isn't 'derived from' the Linux kernel and isn't subject to the viral parts of the GPL. And, all of this is the greyest of grey area. Towards that end, Nvidia has been committing to Nouveau lately for Tegra chips because I think they realize how precarious their legal situation is for chips essientially designed to run Linux.
That reasoning doesn't apply to most kernel drivers, and those vendors are just openly in conflict with the GPL.
Literally any driver that has a piece that runs in interrupt context has to have (in part) a kernel mode driver under Linux.
On top of that, GPUs generally have to have a kernel component, even under systems that put as much as possible under user mode like sel4, because they have their own MMUs that can subvert kernel integrity.
Again, yes there are drivers running in the kernel space, yes they are mostly GPL, no Google isn't re-writing them into Binderized HALs, they can stay as Passthrough HALs.
Nothing of that prevents that all new drivers, except for the ones marked as legacy, are required to be Binderized.
The kernel code of a driver from OEMs allergic to GPL, is hardly different from a signal handler, implementing the minimal set of kernel code and deferring everything else to userspace.
Ah appealing to authority as last argument, naturally any kind of opinion and knowledge of what I might have as Android developer, or former professional experience in systems programming is worthless.
Linux has countless LKMs (modules that run in the kernel, but are not -- broadly -- bound by the GPL). Many, many are closed-source and proprietary. Others aren't because they don't have to be.
You hinted that this transgresses the GPL, and there are those that argue that virtually anything transgresses the GPL. But a lot of very large corporations say otherwise, and there have been zero successful challenges against it.