[n2d4]

There's at least an argument to be made that this is because all the models are heavily trained on GPT-4 outputs (or whatever the SOTA happens to be during training). All those models are, in a way, a product of inbreeding.

[fragmede]

But is it the kind of inbreeding that gets you Downs, or the kwisatz haderach?

[batshit_beaver]

Yes

[pram]

Consider the bulldog: https://youtube.com/watch?v=hUgmkCgMWbg

[sumo43]

Maybe true for instruct, but pretraining datasets do not usually contain GPT-4 outputs. So the base model does not rely on GPT-4 in any way.