[Timpy]

A lot of training data was curated in Kenya[0]. I would imagine if LLM data was curated in Japan our LLMs would sound a lot like the authors of their most popular English text books. Maybe other common Japanese idioms would leak in to the training data, like "ね" or "でしょう", ChatGPT would say "Don't you agree?" at the end of every message.

[0] https://www.theverge.com/features/23764584/ai-artificial-int...

[erikig]

The Indian-born textbook author mentioned (Malkiat Singh [0]) had an inordinate influence on many Kenyan students because his textbooks were the de-facto standard for years. Its interesting how this influence extends as his students get to curate the LLMs on which the world has come to rely.

[0] https://en.wikipedia.org/wiki/Malkiat_Singh

[jojobas]

So twists of training data procurement bring us the best of doing the needful through Africa.

[m4rtink]

You are completely right dajou~ ^_^ !

[delis-thumbs-7e]

Maybe we all should start writing Japanglish to show our authenticity? Or rather, ”Maybe we all should start writing the Japanglish, so that peoples can feel our real soul, you know?”

[bakugo]

I guess it can't be helped.

[koakuma-chan]

It's not because I like you or anything.

[bpodgursky]

This is a wild misunderstanding of LLMs. Data labeling has nothing to do with generating the astronomical text corpus used to train modern LLMs.

[heavyset_go]

The HF part of RLHF to refine the output of LLMs also happens in these places

[astrange]

Note RLHF can only perform selection on existing model outputs, adding new data is SFT or else just more pretraining.

ChatGPT speaking African English was mostly just 3.5. 4o speaks like a TikTok user from LA. 5 seems kind of generic.

[casey2]

樣 is just setting us up for

ChatGPT :|

ChatGPT (japan) XD