|
|
|
|
|
|
by makin
933 days ago
|
|
|
There is a theory that GPT-4's secret sauce is a combination of commissioned high quality training data (which they don't really hide) and an unknown implementation of Mixture-of-Experts [1]. No proof, of course, as the other comments have said, they won't share. The Open AI name is still ironic. I've also still not seen another successful attempt of MoE by any other company, which you would expect to if it was true. [1] https://hkaift.com/the-next-llms-development-mixture-of-expe... |
|
|