[rawoke083600]

Maybe this is an example of where you need an "extra specialized skill"(arithmetic) vs the general and semi-ambiguous-skill of language+conversation.

GPT-3 is "good with conversation (language)"

GPT-3 now needs a "sub-nn-model" to do the very 'specialized skill called math'

*GPT-3 Should 'learn' to recognize which questions should be delicate to a submodel.

[machiaweliczny]

I think this is idea of Google Pathways (Multitude of Expert model). I mean it already works like that in every model but I think they train it differently to have it more separated.