[macNchz]

Depends on the nature of the content you’re working with, but I’ve had some good results using an LLM during indexing to generate a search document by rephrasing the original text in a standardized way. Then you can search against the embeddings of that document, and perhaps boost based on keyword similarity to the original text.

[mrshu]

This is also often referred to as Hypothetical Document Embeddings (https://arxiv.org/abs/2212.10496).

[adamgordonbell]

Do you have examples of this? Please say more!

[joerick]

Nice workaround. I just wish there was a less 'lossy' way to go about it!

[jacobr1]

Could you explicitly train a set of embeddings that performed that step in the process? For example which computing the loss, you compare the difference against the normalized text rather than the original. Or alternatively do this as a fine-tuning. Then you would have embedding that optimized for the characteristics you care about.

[hobs]

Normal full text search stuff helps reduce the search space - eg lemming, stemming, query simplification stuff were all way before LLMs.