[kragen]

Which incentivized human persuaders? Are we talking about top salespeople and litigators, or are we talking about average college freshmen?

It says they recruited participants from the US through Prolific and paid them £10.12 per hour, so probably more like the latter.

[lordofgibbons]

Does it matter? the difference is only 6 months of LLM progress.

[kragen]

It matters if studies like this matter, that is, it matters to people who are interested in what has currently happened rather than what might happen in the future. 6 months of LLM progress keeps not looking like what I expected.

On the other hand, if you're content with your pre-existing predictions about what would happen, which I think is actually a reasonable position, there's no reason to read the paper.

[fzzzy]

Is progress faster or slower than you expected?

[kragen]

An astounding amount of both.

[fzzzy]

That makes sense. Progress does seem very lumpy. Commercialization and secrets means that what we see publicly may not be state of the art. And I certainly never thought that Zuckerberg would be the one providing open model weights. It's a strange new world.