[mjburgess]

It would be bizarre if they benchmarked the models based on actual task performance?

[brookst]

Let’s not move the goalposts. It would be bizarre if they got into a sociological study of impact on job roles.

[spuz]

Not sure why you think that? One of their biggest sales pitches to businesses is the potential for their products to replace certain job roles in the future. Why wouldn't they be actively doing research on real-world usage and impact right now?