[TeMPOraL]

It could be better than what we have now, assuming somewhat smart AI. Right now, data gives plausible deniability, and is the universal scapegoat - you get to do whatever you want, backing this up with misapplied A/B tests and telemetry, with zero statistical rigor. And if someone complains about your decisions, you point to these tests and that telemetry, and cry "we're a data driven company, and the data proves we're right".

[xg15]

I mean, it would help to be honest about the metrics that the A/B tests are evaluated against. If you optimize purely for growth and engagement, then your test results will reflect that - even moreso if your tests are scientifically sound.

I'm continually surprised how companies manage to sell "let's try to keep users glued to the screen as long as possible" as doing something for the general good.

[adrianN]

I have little hope that the people not understanding their data today and lacking any statistical rigor will be able to train an AI to do better. Figuring out whether an AI gives you garbage results seems to be a harder problem than figuring out whether your own results are garbage.