you mentioned the prohibitive size of the vectorizations of documents -- what role, if any, have matrix/tensor decompositions or tensor networks played in helping the search community with this?
I've never seen Tensor Networks used in search, but it wouldn't surprise me if Google, Microsoft, Yandex, and other web-scale search engines were using them, because at their scale it's extraordinarily important for size reduction. We know Google is using BERT (or a newer variant) in production, so they must be doing all kinds of things to lighten the load.
From my perspective of usually dealing with much smaller corpuses, SBERT helps with in a couple ways here practically: it reduces size requirements by an order of magnitude, and also represents sentence context easier than blending individual tokens - which IMO more closely matches the goal of information needs with longer queries.
From my perspective of usually dealing with much smaller corpuses, SBERT helps with in a couple ways here practically: it reduces size requirements by an order of magnitude, and also represents sentence context easier than blending individual tokens - which IMO more closely matches the goal of information needs with longer queries.