[kjfaejgoaeijhei]

To quote the article: "Chatbots were nearly five times more likely to produce broad generalisations than their human counterparts."

The following might be rude and unkind, but at this point frankly necessary: LLM apologists should stop projecting.

[ekianjo]

> Chatbots were nearly five times more likely to produce broad generalisations than their human counterparts

Never seen a journalist not make broad generalizations on a scientific study so I am not sure where these numbers are coming from.

[kjfaejgoaeijhei]

You can read the original research article here: https://royalsocietypublishing.org/doi/10.1098/rsos.241776

It will tell you exactly where the numbers are coming from - the comparison is between (a variety of) LLMs and "expert-written summaries from NEJM Journal Watch".

[redcobra762]

What is the human counterpart to an LLM? The article doesn't describe.

And "LLM apologists" is so polemical it's hard to take seriously. We get it, you don't like GenAI. That's fine, but can we talk about it without getting normative?

[kjfaejgoaeijhei]

The article doesn't, but you can go to the "more information" section and keep digging:

"To systematically assess differences between LLM-generated and human-written summaries, we also collected the corresponding expert-written summaries from NEJM Journal Watch (henceforth ‘NEJM JW’)"

[bethekidyouwant]

Right so not scientific American how convenient (and may I add scientific!)