Yes, but those modules need to be maintained as the kernel is updated. If a specific architecture’s tree is abandoned, it would make sense to remove it from the tree until someone picks it up again.
The problem: once something is removed from the tree, it is almost impossible to get it back in, since whoever wants to do that has to replay the extreme amounts of refactorings done since the removal.
The Linux kernel famously only guarantees outside stability of interfaces, but in-kernel interfaces are regularly changed - one might suspect intentionally, to make the lives of people doing off-kernel work (such as graphics card drivers) as difficult as possible so that they will eventually open source their code and ship it in the kernel.
How’s that any different than keeping it updated while it’s in the tree? If it’s stale and someone comes along to fix it, it’ll be in the same state it was when it would’ve been removed. Stale code needs to be updated regardless of where it came from.
The Linux kernel famously only guarantees outside stability of interfaces, but in-kernel interfaces are regularly changed - one might suspect intentionally, to make the lives of people doing off-kernel work (such as graphics card drivers) as difficult as possible so that they will eventually open source their code and ship it in the kernel.