[mjburgess]

It's strange that none of these $100s bn+ companies fund empirical research into the effects of AI tools on actual job roles as part of their "benchmarks". Oh wait, no its not.

[brookst]

Agree, it would be bizarre if they did.

[mjburgess]

It would be bizarre if they benchmarked the models based on actual task performance?

[brookst]

Let’s not move the goalposts. It would be bizarre if they got into a sociological study of impact on job roles.

[spuz]

Not sure why you think that? One of their biggest sales pitches to businesses is the potential for their products to replace certain job roles in the future. Why wouldn't they be actively doing research on real-world usage and impact right now?