| I always wonder how those statistics match reality. What we want to know is the integral of U(t) for t in [0, inf), the rate of users joining at time t integrated across all time. But U(t) = H(t) + E(t), where H is hard to measure (feedback, effects over time, etc) and E(t) is some "simple" to measure function. What people actually measure is E(t) over a short time window, say t in [0,1], because H(t) more or less contributes a constant in such a small window. This is then used as a proxy for U(t), to adjust their strategy. The problem comes in that H(t) might dominate E(t) over the "lifetime" integral of U(t), while seeming irrelevant in the short term you can measure. (Think users from establishing a reputation.) If your optimizations from your measurements increase E(t) at the cost of H(t) (as they often do -- knowingly), then it's possible you've actually decreased the "lifetime" integral of U(t) while making all "positive" adjustments. It's the stats equivalent of leaving the trail up a mountain that passes through foothill valleys -- just going "up" without a plan is unlikely to climb a mountain, you need some broader vision. |