Would be interesting to see how a specialised model does against a model with more params, I use qwen8b in my HA and would love to downgrade to 1.7B and optimise HA... But you have no stats or comparisons
Definitely! We’re working on a benchmark in the coming weeks, and although we don't have data to support against qwen8b if you do try it out please let us know your experience in how it compares.