[kolinko]

I may be wrong but with midern llms biases aren’t really used any more.

[tripplyons]

From what I remember, larger LLMs like PaLM don't use biases for training stability, but smaller ones tend to still use them.