|
|
|
|
|
|
by chaxor
1057 days ago
|
|
|
This will perform worse in many cases, better in some cases. There is a lot of knowledge that can be transferred between datasets. For example, "describe to me if this Amazon product is likely to have stronger tensile strength and if its materials are more safe?" requires knowledge not only from a database of Amazon products and their descriptions, but in this case leaving out knowledge from physics textbooks could be detrimental.
Ultimately, these are the types of problems we want these systems to excel at as well, so it's important to access all of the training data. MoE is still a decent idea (can help transfer some of the knowledge between models with a model on top of others), but in order to not get wildly conflicting and/or unrelated stories from each model, some overlap is needed to provide a clearer story to the top model. |
|
|
If A answers "This toaster is made of plastique and paper, one would have to look up their tensile strength to answer your question"
And B answers "I don't know what materials this toaster is made of, but the best tensile strength in toasters is reached when using iron, ok tensil strength is achieved by using copper. One should avoid plastique and paper as these have very bad tensil strenght"
Then C could imply that the tensil strength of that toaster is not good.