[RohMin]

I haven't been able to get a great answer regarding why OpenAI is consistently leading the pack. What could they possibly be doing different? I can't imagine they've invented a technique that nobody else can reach at this point

[autokad]

my guess is openai spent the most human hours fine tuning the model, and other companies are running into problems and trying to deal with them whereas openai already learned those lessons a long time ago

[dontupvoteme]

Human hours, aka poorly paid contract workers in Africa.

[lolpanda]

This is not true. For LLM data labeling, the knowledge workers are very well paid. The hourly rate is way above minimum wage. The questions oftentimes require domain knowledge. They are complex enough and cannot be answered by random person on the internet. AFAIK most of them are located in US.

[dontupvoteme]

Wasn't there some dubious history with OpenAI and poorly paid workers in the third world?

[BryanLegend]

There's a network effect in that they are used more so they've generated more feedback from users, which is then used to improve GPT.

[vitorgrs]

That doesn't make sense because GPT4 finished training in August 2022 - before ChatGPT 3.5 release in November 30.

You could say they got data to train RLHF after the training, but that seems unlikely.

Bing was launched in February 7 with GPT4 - that's just 2 months after ChatGPT launch.