|
|
|
|
|
|
by reinitctxoffset
11 days ago
|
|
|
Well, since we started training them on RLHF-style rating of the last generation they've totally collapsed back onto predictable suffix generators, which is why there are desperate, low-AUC BERT-inspired clasufiers bolted to them now. They have like, Chomsky grammars now. And you can't fix it, it's structural to the process. You have to rewind almost to the pre-train. |
|
|