[Ozzie_osman]

Yeah, golden cohorts can work, but they are really hard to pull off, especially for logged out traffic (which is where you'd use most of these patterns anyway). Good luck tracking me over 6-24 months across different devices and locations. And cross-contamination is hard to prevent (for instance, the golden cohort might suffer from global effects like worse content due to loss of power users or even from stumbling across brand-damaging threads like this). It also just adds a lot of product complexity to keep behavior around that long.

That said, they can work. Twitter famously did something like that for their time-based vs algorithmic feed and I think YouTube does it pretty regularly.

The biggest issue, though, is that by the time you get results from any long-term experiments, most of the decision-makers (PMs, EMs, etc) have probably moved on away having taken credit for the short-term wins they delivered.

[wting]

The person who created Twitter's experimentation platform is also at Reddit, and heavily influenced Reddit's experiment design and review process.

But yes, a revolving door of product leaders and decisions is going to bias towards short term optimization.

[nwienert]

It’s shocking to me how people sell out like this. You have to know deep down that all these hostile short term juicers destroy the brand, each malfeasance creating more room for a competitor. I mean you guys replaced Digg, cmon.

The audacity to claim “it works”, in italics no less.

The real shame of the current tech companies is they have no principles, no long term vision. They all feel like they follow the same curve, a bunch of managers hitting KPIs during their 2-5 year stint before trading up, ending in some PE firm diving in at the end for the final squeeze.

They’re lemons being juiced dry, when they should be a garden of lemon trees.

“But we got 20% more juice than last year!!”

Yea, you did.

[MomoXenosaga]

It is not shocking when you realize it is about self-interest. You get your bonus and salary in the short term who really gives a shit what happens 7 years from now?

[nwienert]

Not shocking for the managers, just sad. Shocking that executives don’t get it so consistently.