[Sol-]

How does it perform on HuggingFaceExploit bench? Suspiciously absent, so not sure if I can take the model seriously.

On a serious note, I hope they improved their extremely sabotaging and unspecific bio safeguards, which prevented Fable from being used in any codebase that ever so slightly grazed medical terminology or data and made me switch to 5.6 Sol.

[rdedev]

My codebase had a dataset with a bunch of SMILES strings and the word Malaria. Fable did not want to touch that codebase

[sivakon]

What is HuggingFaceExploit bench?

[scriptsmith]

It's a reference to this story where an OpenAI model broke out of its sandbox during cyber benchmarking and hacked into HuggingFace, in order to obtain test solutions: https://news.ycombinator.com/item?id=48997548