[Roark66]

I have to say in my experience falcon-40b-instruct got very close to chatgpt (gpt-3. 5),even surpassing it in few domains. However, it is important to note (not at all)OpenAI are doing tricks with the model output. So comparing OS models with just greedy output decoding (very simple) is not fair for OS models.

Still, I'm very excited this model at 13B seems to be matching falcon-40B in some benchmarks. I'm looking forward to using it :-)

[fnl]

> OpenAI are doing tricks with the model output

Do you have any pointers to the “tricks” that are being applied?

[jcuenod]

Sounds like a reference to Mixture of Experts

[zzzzzzzza]

could be something like prompt rewriting or chain of thought or reflexion going on in the background as well