[missedthecue]

This conversation always gets lost in the weeds. You don't need a SOTA model to replace an entire engineer role from the time he gets into the office until he leaves.

If you have a team of 10 and make them all a little faster, you can do the same amount of work with 9. Run this out over the entire industry and it's hundreds of thousands of roles that are redundant.

[angoragoats]

Do you have any research, empirical data, or other hard evidence that shows that a model (especially a non-SOTA model) makes engineers even "a little faster"? I'm aware of anecdotes, but nothing more.

[missedthecue]

The cleanest positive study is Cui et al., because it used randomized rollout in real companies. Developers at Microsoft, Accenture, and a Fortune 100 electronics manufacturer were randomly given access to an AI coding assistant. Pooled across 4,867 developers, the authors estimate a 26.08% increase in completed tasks among users. They also report increases in commits and builds.

While that research was published in 2026, it analyzed the 2023 period. Very much NOT sota models!

https://pubsonline.informs.org/doi/10.1287/mnsc.2025.00535

[angoragoats]

Thank you!