[dragonwriter]

> GPT 4 is based on the same architecture, but at 8*222B.

Do we actually either no that it is MoE or that size? IIRC both if those started as outsidr guesses that somehow just became accepted knowledge without any actual confirmation.

[moffkalast]

Iirc some of the other things the same source stated were later confirmed, so this is likely to be true as well, but I might be misremembering.