Adding a cross-encoder to your app means including pytorch/transformers and a model as a dependency. For people using OpenAI or Cohere embedding apis and lightweight infrastructure this can be a big pain.
Proposed architecture doesn't limit you to use self-hosted transformers only, you can use OpenAI just as easily. And you don't need a to install a "module" for that