[lutusp]

> What makes trading any harder than programming or engineering or abstract mathematics or physics?

That's easy to answer -- if you're very smart, you will do better in math and physics, and the positive correlation between intelligence and success actually has meaning. But in equities trading, being very smart does no good at all, and might even represent a handicap. The reason? Equity pricing is mostly random noise, very high entropy.

Many very smart people have tried to model and then beat the market, and (apart from chance outcomes) no one has succeeded in the sense that they located a reliable, describable "method" for it.

Also, if in principle someone discovered an actual winning system, by publishing the system he would destroy its effectiveness (because once a "system" is put into practice, once everyone uses it, it loses its effectiveness).

Obviously for a random collection of investment schemes, half will do better, and half worse, than the average market, but the half that do better will naturally enough claim the outcome resulted from their investment genius -- even if they're smart enough that they should know better.

I can use a computer to model a random market and random investors, using random price changes and random trades, and not surprisingly, half the investors will do better than the average outcome. What is surprising is that, when the above-average investors are humans, they invariably try to claim this chance outcome resulted from a winning system that they will sell to you for some princely sum.

More here: http://arachnoid.com/equities_myths

> Pointing that out X is hard and most people fail is a tautology, it's true of all difficult professions.

Yes, but there's a qualitative difference between "hard" and "impossible".

[throwawaymsft]

You're basically saying investing is random chance. Ok.

What is the p-value that Warren Buffett's random investment plan outperforms the market 39/47 years?

http://finance.fortune.cnn.com/2012/02/25/buffett-berkshire-...

If a drug beat a placebo in 39/47 experiments, would it be approved by the FDA? :-)

There are investors who get lucky, and there are investors with sound strategies. The notion that every winning strategy gets immediately and flawlessly applied by every actor in the market is a fantasy world populated by homo economicus.

[lutusp]

> You're basically saying investing is random chance.

No, what I am saying is that random choices can produce the illusion of successful investment strategies for 1/2 the practitioners. This isn't at all controversial -- it's equivalent to saying that half of people are above average in intelligence.

> What is the p-value that Warren Buffett's random investment plan outperforms the market 39/47 years?

That's not a particularly interesting question (and it's not an improbable unbroken streak of 39 years, but a mixture of successes and failures, much more likely to result from chance). What is more interesting is to ask what Buffett's market performance would have been without the announcement and herd effects? Everyone wants to invest in the same stocks Buffett invests in, and Buffet's investments are public knowledge. The herd can be relied on to make Buffett's choices after he has established his position, which means he's positioned to benefit from the public's responses to his choices.

Again, I never claimed that investing is random choice. I only said that random choices work for 1/2 the investors, and those investors are likely to attribute their success to genius instead of chance.

[alsocasey]

Perhaps more to the point: Buffet's results are unlikely (though likely the result of many different strategies, some successful, others not...), but in a market with so many investors, it is perhaps not so surprising that one so successful arose?

[lutusp]

> Buffet's results are unlikely (though likely the result of many different strategies, some successful, others not...), but in a market with so many investors, it is perhaps not so surprising that one so successful arose?

Quite correct, in fact, in an equities trading community the size of the present one, successes like Buffet's can and do result from chance. This is not to claim that any particular one does result from chance, only that this possibility is the first one that a scientist or statistician would need to consider.

[gnaritas]

> Buffet's results are unlikely

That wasn't Buffets point; his point is that those who followed certain strategies of value investing consistently win and he gave examples. That's more than a random outcome.

[lutusp]

> That wasn't Buffets point; his point is that those who followed certain strategies of value investing consistently win and he gave examples.

"Consistently win" is obviously false. If someone had an actual method (not a random unexplained event) with a description, that could be tested and that could consistently beat market averages, it would surely be applied and the market would collapse. The market didn't collapse, so there is no such method.

How is that so hard to figure out? There is no winning strategy, no secrets of the winners. There are people who make more than others in equities, but not because of a describable, scientifically testable system.

When a scientist encounters a description like this, he always assumes a priori that it's chance. That agrees with the null hypothesis and Occam's razor. It also keeps people from selling him worthless investment books.

[throwawaymsft]

You're restricting yourself with the giant caveat that the system must be perfectly deterministic (i.e. capable of being executed by code). Why? For the convenience of your testing?

An analogy: humans have beaten computers at Go for decades. Is the human strategy random chance? It must be, since they cannot write down their algorithm so well that anyone else can be a Go master!

Go strategy (similar to investment strategy) cannot be perfectly laid down (though the general principles can), and yet the results are clear. Would you argue there is nobody inherently better at Go than anyone else? Even if they are the best performing Go-perfomer ever, who has beaten every computer system for decades and decades?

I'm not sure how much of a butt-kicking Buffett needs to perform against the market for you to believe there may be an element of skill at play.

[gnaritas]

> it would surely be applied and the market would collapse. The market didn't collapse, so there is no such method.

You keep saying that, but it's full of assumptions that simply aren't true. Just because a successful method exists does not mean everyone can or will apply it nor does it mean the market will collapse.

> How is that so hard to figure out?

How it it so hard for you to see the absurdity of what you're saying?

> When a scientist encounters a description like this, he always assumes a priori that it's chance. That agrees with the null hypothesis and Occam's razor.

There's a reason scientists are in general not rich people. And I have much respect for scientists, but you're just being absurd.

Investigate a bit before just claiming it's nonsense because you're not giving it the thought you should, you're just presuming you're right and making absurd claims based on absurd assumptions that don't hold water.

http://www.businessinsider.com/warren-buffett-on-efficient-m...

[throwawaymsft]

> That's not a particularly interesting question (and it's not an improbable unbroken streak of 39 years, but a mixture of successes and failures, much more likely to result from chance).

Why is any particular permutation of 39 wins and 8 losses more improbable than another? That's like saying HHHTT is more likely than HTHTH.

> The herd can be relied on to make Buffett's choices after he has established his position, which means he's positioned to benefit from the public's responses to his choices.

But now you're playing it both ways. Are markets efficient or not? Can't every rational investor realize the herd will do this and just ape Buffett?

I may have misunderstood, but your thesis seems to be that markets are efficient, and no strategy can consistently beat the market over time. My thesis is that we have an existence proof that isn't the case.

[lutusp]

>> That's not a particularly interesting question (and it's not an improbable unbroken streak of 39 years, but a mixture of successes and failures, much more likely to result from chance).

> Why is any particular permutation of 39 wins and 8 losses more improbable than another? That's like saying HHHTT is more likely than HTHTH.

Again, that is not what I said. Are you trying to misinterpret, or is this all inadvertent? Had I anticipated your effort to misinterpret, I would have said that a random sequence of 39 heads is less probable than three sequences of 13 heads, each separated by some other unidentified, random sequences or, as I put it originally, "a mixture of successes and failures".

>> The herd can be relied on to make Buffett's choices after he has established his position, which means he's positioned to benefit from the public's responses to his choices.

> But now you're playing it both ways.

Nonsense. I'm explaining how any outcome can be attributed to chance, not that they are explained by chance.

> Are markets efficient or not?

That's not even a topic, and if it were, it wouldn't be a binary choice.

> I may have misunderstood,

You thoroughly misunderstood, no ambiguity. And no offense meant.

> ... but your thesis seems to be that markets are efficient, and no strategy can consistently beat the market over time.

No to the first (no one knows) but absolutely yes to the second -- no strategy can consistently beat the market over time. Isn't that obvious? Any strategy that consistently beat the market, and that could be expressed as a deterministic algorithm, and that was something other than a transient effect of no real value, could be used to drain the market of its capital in a matter of months -- and therefore it would be. The market would collapse and businesses would refuse to trust equities.

My point? If there really was such a strategy, it would demolish the market, meaning there would be no remaining market to beat consistently. The golden goose would lie dead.

Just think a bit more deeply.

[throwawaymsft]

I appreciate the discussion, let me clarify.

> a random sequence of 39 heads is less probable than three sequences of 13 heads, each separated by some other unidentified, random sequences

Investors are not going for streaks. They are going for overall return. A better way to put it: what are the chances that over a 48-year timeframe, a random strategy will return ~20% annualized return compared to the market's ~10%? Ignore an arbitrary year limitation (why not measure monthly or daily return?). What is the chance a random strategy would outperform so long?

Or, a better question: what is a chance that a random strategy will be the best-performing investment over a 30-year period (1926-2011), and the subject of academic studies:

http://www.econ.yale.edu/~af227/pdf/Buffett's%20Alpha%20-%20...

Buffett is an existence proof that investment is skill, not chance. He created the 9th most valuable company in the world. You can believe that was entirely due to chance if you wish.

(On beating the market consistently: you don't have to crush the market to extract value. Certain strategies only work for certain amounts of capital. You can be an Apex predator and not drive the entire ecosystem into extinction.)

[lutusp]

> A better way to put it: what are the chances that over a 48-year timeframe, a random strategy will return ~20% annualized return compared to the market's ~10%?

Easily answered. Let's say that a 10% return is the mean return, and one standard deviation is 5% -- just an example, and these numbers aren't real (although they could be established by asking everyone what their returns are).

So a return of 20% or better represents two standard deviations above the norm, or 2.2% of the investing population (this is a one-tailed distribution). How many investors will achieve that result in a large population? 2.2%. In a pool of a million investors, that's 22,000 people.

> Buffett is an existence proof that investment is skill, not chance.

With all respect, it's more accurate to say that your view of probability is an existence proof that many people don't understand statistics. Let me ask you -- do you understand how science works? Scientists don't say what you just said, ever. They say that the probability that this outcome resulted from chance is p ("p-value"), referring to the probability that the result arose because of chance. (My 2.2% probability above is a p-value.)

When the LHC scientists announced that they believed they might have detected the Higgs boson, did they say that their measurement constituted an "existence proof" that the Higgs was real? No, because they were scientists addressing educated people, they expressed their result in terms of a p-value -- p was the probability that their result came about because of chance, not the hypothesized particle.

A chance result isn't the last possibility that a scientist considers, it's the first. And no one who has been educated claims that a result that might have arisen by chance constitutes an "existence proof".

[zeckalpha]

That's a fundamental misunderstanding of random.

Investing being random chance doesn't mean everyone will do about average. It means that some people will make quite a bit of money and some people will lose quite a bit of money. It just happens that Warren Buffet is an outlier.

[throwawaymsft]

I'm not sure I understand. Buffett has a strategy X, which has defeated strategy Y (buy a market index fund) 39/47 times.

We can say any return in strategy X is

1) Due to random chance (i.e., it defeated the market 39 times "randomly") 2) Not due to random chance, and there is an inherent advantage

I'm saying the probability of 1) is sufficiently low so as to believe 2).

[lutusp]

You're missing the point. What is the probability that someone flipping a fair coin will flip 20 heads in an unbroken sequence? The answer is 2^-20 = about 9.5 * 10^-7.

Next question. How many people need to be flipping coins for one of them to have a better than even chance to flip 20 heads in a row? Answer: about 3/4 million.

Next question. How many investors are there in the world? Answer: many more than 3/4 million.

Next question. If someone among millions of investors makes 20 successful market picks in an unbroken sequence, what's the probability that he will attribute that outcome to blind chance, and what is the probability he will start selling a book titled "Secrets of the Winners" on late night TV?

My point? When confronted by an unexplained occurrence, it's wise to consider the possibility that it's a random outcome. This is called the "null hypothesis" and it's the first possibility a scientist considers.

[Thrymr]

The question is, how many potential Warren Buffets were there in the pool to start with? You can't ask the question "what are the chances that this particular outcome would happen?" with a sample of one, you have to use the population it is drawn from. Otherwise it would be like saying "That's amazing! The lottery numbers today were 56 23 45 12 27 91! What are the chances!" Exactly the same as any other combination (very, very low). The trick is picking them in advance.

[zeckalpha]

While the probability of any individual defeating the market is 0.5 in any given year, the probability of someone from the pool of N investors getting beating the market 39/47 years is N * 0.5 ^ 47. (It's been a while since I've done probability, so correct me if I'm wrong.)

[lutusp]

> the probability of someone from the pool of N investors getting beating the market 39/47 years is N * 0.5 ^ 47.

No, that's wrong. If the original performance had been an unbroken sequence of successful years, say, 39, it would be possible to apply the binomial theorem to it, but the binomial theorem would need to be applied both to the original probability and to the calculation of how many investors would be needed to create a better-than-even prospect of that outcome.

But the 39 successful years were randomly distributed among years where the performance wasn't better than market indices, which makes the probability much higher for it being explainable by chance -- how much higher depends on the actual pattern, which I wasn't able to find.

[magnusjonsson]

(47 choose 39) * 0.5 ^ 39 * 0.5^8 = 0.0000022, so that'd be about 1 in 500000 of all who traded for this long.

[gnaritas]

> Many very smart people have tried to model and then beat the market, and (apart from chance outcomes) no one has succeeded in the sense that they located a reliable, describable "method" for it.

That's an easy claim to make when you simply dismiss anyone who succeeds as random chance.

> by publishing the system he would destroy its effectiveness

Obviously and not disputed.

> I can use a computer to model a random market and random investors, using random price changes and random trades, and not surprisingly, half the investors will do better than the average outcome. What is surprising is that, when the above-average investors are humans, they invariably try to claim this chance outcome resulted from a winning system that they will sell to you for some princely sum.

That's not damning evidence, merely circumstantial.

> Yes, but there's a qualitative difference between "hard" and "impossible".

Yes there is, however I don't believe it's been adequately proven to be impossible.

[lutusp]

> That's an easy claim to make when you simply dismiss anyone who succeeds as random chance.

I never said this anywhere. You're confusing the practice of science with a blanket statement I never made. Pretend to be a scientist -- prove me wrong. Locate where I said what you claim I said.

> Yes there is, however I don't believe it's been adequately proven to be impossible.

Oh, great -- "proven to be impossible". Before you go on, learn about the impossibility of proving a negative:

http://en.wikipedia.org/wiki/Russell's_teapot

Quote: "Russell's teapot, sometimes called the celestial teapot or cosmic teapot, is an analogy first coined by the philosopher Bertrand Russell (1872–1970) to illustrate that the philosophic burden of proof lies upon a person making scientifically unfalsifiable claims rather than shifting the burden of proof to others ..."

Remember -- all I have ever said is that the possibility of a chance outcome cannot, must not, be dismissed, and to a scientist, it is the first possibility to be considered, not the last.

> That's not damning evidence, merely circumstantial.

What? According to the scientific rules of evidence, the burden of evidence is not mine to prove that a winning stock picking strategy does not exist (i.e. prove a negative), the burden is on others to prove that it does.

[gnaritas]

> I never said this anywhere. You're confusing the practice of science with a blanket statement I never made.

Poorly worded, I wasn't claiming you said it, I was saying you could claim it and you would because you consider it the null hypothesis.

> Before you go on, learn about the impossibility of proving a negative:

I don't need to research Russell's teapot, I'm well aware, that was a poorly thought out statement on my part is all. I didn't mean to ask you to prove a negative.

> Remember -- all I have ever said is that the possibility of a chance outcome cannot, must not, be dismissed, and to a scientist, it is the first possibility to be considered, not the last.

It's not being dismissed. The issue is whether it's just chance; that Buffet's disciples have also done consistently well disputes the null hypothesis. Please actually read what Buffet has to say[1], he's not stupid and he's not ignoring the null hypothesis.

[1] http://www.businessinsider.com/warren-buffett-on-efficient-m...

[gnaritas]

Excellent reply.