Hacker News new | ask | show | jobs
by RyanAdamas 702 days ago
Personally, language diversity should be the last thing on the list. If we had optimized every software from the get-go for a dozen languages our forward progress would have been dead in the water.
2 comments

You'd think so, but 3.5-turbo was multilingual from the get go and benefitted massively from it. If you want to position yourself as a global leader, then excluding 95% of the world who aren't English native speakers seems like a bad idea.
Yeah clearly, OpenAI is rocketing forward and beyond.
Constant infighting and most of the competent people leaving will do that to a company.

I mean more on a model performance level though. It's been shown that something trained in one language trains the model to be able to output it in any other language it knows. There's quality human data being left on the table otherwise. Besides, translation is one of the few tasks that language models are by far the best at if trained properly, so why not do something you can sell as a main feature?

Language diversity means access to more training data, and you might also hope that by learning the same concept in multiple languages it does a better job of learning the underlying concept independent of the phrase structure...

At least from a distance it seems like training a multilingual state of the art model might well be easier than a monolingual one.

Multiple input and output processes in different languages has zero effect on associative learning and creative formulation in my estimations. We've already done studies that show there is no correlation between human intelligence and knowing multiple languages, after having to put up with decades of "Americans le dumb because..." and this is no different. The amount of discourse on a single topic has a limited degree of usability before redundancies appear. Such redundancies would necessarily increase the processing burden, which could actually limit the output potential for novel associations.
Google mentioned this in one of their papers, they found for large enough models including more languages did indeed lead to an overall increase in performance.
Considering Googles progress and censorship history, I'm inclined to take their assessments with a grain of salt.
Humans also don't learn by reading the entire internet... assuming human psych studies apply to LLMs at all is just wrong.