Hacker News new | ask | show | jobs
by whimsicalism 760 days ago
They have not. You probably read "MoE" and some pop article about what that means without having any clue.
1 comments
matsemann 760 days ago
If you know better it would be nice of you to provide the correct information, and not just refute things.
link
whimsicalism 760 days ago
gpt-4 is a sparse MoE model with ~1.2T params. this is all public knowledge and immediately precludes the two previous commentators assertions
link