|
|
|
|
|
|
by PaulHoule
777 days ago
|
|
|
Some of the SBERT models now are based on T5 and newer architectures so there's not. The FlagEmbedding model that the author uses https://huggingface.co/BAAI/bge-base-en-v1.5 is described as an "LLM" by the people who created it. It can be used in the SBERT framework. I tried quite a few models for my RSS feed recommender (applied after taking the embedding) and SVM came out ahead of everything else. Maybe with parameter tuning XGBoost would do better but it was not a winner for me. If you look at the literature https://arxiv.org/abs/2405.00704 you find that the fashionable LLMs are not world-beating at many tasks and actually you can do very well at sentiment analysis applying the LSTM to unpooled BERT output. |
|
|
Oh thanks! Right I had heard about T5 based embeddings but didn't realize it was basically an LLM.
> I tried quite a few models for my RSS feed recommender (applied after taking the embedding) and SVM came out ahead of everything else. Maybe with parameter tuning XGBoost would do better but it was not a winner for me.
XGBoost worked the best for me but maybe I should retry with other techniques.
> you find that the fashionable LLMs are not world-beating at many tasks and actually you can do very well at sentiment analysis applying the LSTM to unpooled BERT output.
Definitely. Use the right tool for the right job. LLMs are probably massive overkill here. My non-LLM based embeddings work just fine for my own recommender so shrug.