[mixmax]

using past performance to estimate future performance simply does not work

This is not always true.

I had a theory a few years ago that went like this: The price of a stock is a direct function of the perceived price of the company multiplied by a risk factor: The more risk the less the stock is worth. On the day the yearly report is publicised for a company the risk is big just before the publication because nobody knows for sure what the numbers are, and low just after the publication because everybody now knows. So according to my theory the stock price of an arbitrary company should statistically go up on the day that the yearly report is made public.

I, painstakingly, found some historical data on this (it wasn't easy) and found that it was spot on. A bit of statistical analysis showed thet there was a definite gap to be exploited.

A friend of mine showed in his Masters thesis that there were certain patterns that would almost always be present in IPO's that could be exploited if you knew them.

So there are definitely loopholes where past behaviour shows future performance, but not a lot of them.

And professional investors aren't always as smart as they're made out to be. I know a few, and hackers are a lot smarter in regard to numbers.

[mattmaroon]

The problem is that the market is dynamic and chaotic. Measuring past data doesn't give you much indication of how much your algorithm would have made, even in hindsight, because every buy and sell you would have executed would have changed the market and the future. Even relatively small orders can have big ripple effects, especially if they just happen to trigger standing limit orders.

Genuinely measuring the market (by trading) changes it.

[skaylie]

Good point. As pointed out on the rest of this list, you should be very careful about "data mining", where simply finding a strategy that works on historical data may not be replicable. Also, some strategies work well in some markets (like high volatility) and poorly in others. I think a next level of analysis is trying to figure out when to turn particular strategies on or off.

As for the attacks on technical analysis, many practitioners probably are deceiving themselves that they've found some secret sauce. However, since the market is made of 100% of the people who transact in it, the presence of technical traders means that there must be some price movements as a result of these trends. One ideal would be to find a strategy that beats other technical analysts to the punch. A lot of people use the 12/26 MACD, but maybe the 11/25 MACD would help you eat their lunch before they entered the market...

It is all very tricky and takes a lot of discipline to avoid fooling yourself and getting into real trouble. Also, people need to make sure to focus on both sides of the trade - you only book the profit once you've actually sold the position (most people seem to be much more focused on entries than exits)

If anyone is interested in this kind of software but aren't so into learning a new language (perhaps less applicable to this group), you should check out QuantRunner Software http://www.quantrunner.com. In full disclosure, I am the CEO of the company, but we really focus on making this kind of back testing analysis easier for people to do without programming experience. We also have some novel tools for helping people actually improve their strategies, rather than simply iterate tests. If I had the true answers on the best strategies, I probably wouldn't run a software company. All we hope for is to make it easy enough for people to do their homework without having to worry about coding errors as well as poor strategies.

[steveplace]

Ideally, yes.

In practice, no. Trading a couple lots of /es futures will not do much to the market. More like pissing in the ocean. In illiquid markets you can move price, but not the stuff that you would want to blackbox anyways.

[mattmaroon]

It can. If someone has a large limit order and your small order hits their trigger price, whereas the next person might have done the opposite and moved away from it, you could have caused a huge change in the direction of the market. Or the same is true of a cascading series of smaller limit orders, etc.

And since making money is presumably based on volume, if you did small trades, you'd end up doing a lot of them to make it worth your while. But in almost all forms of betting, it's better to make fewer, better wagers.

[steveplace]

I don't know if you've traded /es futures, but that's nearly impossible to do. There's a couple different markets that put in bid/ask, and normally you'll just see the closest spread, but there's usually underlying bids as well. That's why it's so liquid. I'm looking at premarket futures right now and the qty of contracts offered at the bid/ask ranges from 10-100. That means to take the bid out you'd have to have a margin of around 100k.

And since making money is presumably based on volume, if you did small trades, you'd end up doing a lot of them to make it worth your while.

Not true with futures. They are risky, but you can get some pretty nice returns, especially in the volatility that we're having right now. On a single contract you can expect to pull 1k-2k, and that's on one trade per day. In terms of R, you're looking at 10R-20R per trade if you're experienced enough (>2 years).

[bayes]

The trouble with fewer better wagers is that wagers - even very good value ones - can lose. So if you're only making a few of them there's a significant probability of losing overall. Your expected value may be high, but your standard deviation is even higher.

I'd much rather have 10,000 independent bets each with a 0.5% edge than 10 independent bets each with a 20% edge.

[steveplace]

I'd much rather have 10,000 independent bets each with a 0.5% edge than 10 independent bets each with a 20% edge.

Not with trading, you want to cut losses quick and let winners run. The model behind Long Term Capital Management, as well as a ton of quant firms that went under this past Q, was the take-a-bunch-of-trades-for-small-profit... and it works until you get a six sigma event (see 2008).

[mixmax]

Yes, in that respect it's like physics - by measuring you are altering the experiment.

The volume on the day the reports came out was quite heavy, but if you had bought a substantial amount of stock it would certainly alter the market as you say. The point is that I'm pretty certain there are holes that will allow you to look at historic data and make statistically good buys - but you have to look where noone else is looking. Like the correlation of Nokia stock to the Finnish weather. Some of these will be large enough that you can make money, even though you alter the experiment.

On an unrelated note - I just read through your blog and found it very insightful :-)

[mattmaroon]

There's so much money to be made in the stock market (most of the money on Earth) and so many people searching for that data with so much resources (including math PhDs by the hundreds, some of whom I know) that looking somewhere nobody else is is virtually impossible. You're much better off focusing on your career and buying ETFs.

And thanks!

[kingkongrevenge]

forex dwarfs global equity markets.

[volida]

the first rule of trading is that you join the market not try to alter it.

[drusenko]

this is really interesting to me -- i spent a lot of time examining implied and actual volatility with regards to option pricing right before an earnings report. the short and skinny is that (as makes sense) volatility rises right before earnings are released and falls right afterwards. if you know that volatility is going to rise, you could easily plug the rest of the numbers into the black-scholes model and find undervalued options. they weren't always there, but at times, you could find options that would lose less in value due to time decay than they would gain due to the increased volatility with enough volume to not start making the market -- a great arbitrage opportunity.

unfortunately, i didn't have as much time in college to study this as i would have liked, and all of the good volatility data seems to be hidden behind a pay wall (bloomberg stations!)

one thing is for sure: even though the market approximates efficiency, there are still arbitrage opportunities to be found in the right places.

[marvin]

There are probably lots of loopholes, but we don't know about them. It can't be a coincidence that the largest hedge funds have beaten the market spectacularly every year the last 20 years. (And if they put 10% some of those gains in a money market account, they are guaranteed to have made positive returns no matter what happens).

My personal this-is-what-i-think theory is that there are lots of patterns in the pricing of equities and other market components, but if they ever become publicly known, they cease to exist. The best hedge funds (or best traders, take your pick) manage to find patterns that no one else does. This does not protect them from a spectacular implosion if their models for some reason become invalid.

There is a cult of believing in the perfect randomness of market pricing. Saying that market developments are random is practically the same as believing in the efficient market hypothesis. The market cannot be perfectly efficient, because there are always interactions that no one will be able to predict. Most of these interactions are no doubt chaotic (unpredictable), but there is no way that traders instantly discover every pattern that is predictable.

[steveplace]

That risk you speak of may not be discounted in the price, but rather in the implied volatility. You'll see that after major earnings reports/announcements there is an IV crush on the options board.

You can game this by looking at historical IV for pre-earnings and see if the current is over/under. You can then sell straddles or strangles if you think the price will stay in that area.

It's even better for GOOG because they always release earnings right before options expiration, so there's a ton of voodoo going on in their price.