[mattmaroon]

The problem is that the market is dynamic and chaotic. Measuring past data doesn't give you much indication of how much your algorithm would have made, even in hindsight, because every buy and sell you would have executed would have changed the market and the future. Even relatively small orders can have big ripple effects, especially if they just happen to trigger standing limit orders.

Genuinely measuring the market (by trading) changes it.

[skaylie]

Good point. As pointed out on the rest of this list, you should be very careful about "data mining", where simply finding a strategy that works on historical data may not be replicable. Also, some strategies work well in some markets (like high volatility) and poorly in others. I think a next level of analysis is trying to figure out when to turn particular strategies on or off.

As for the attacks on technical analysis, many practitioners probably are deceiving themselves that they've found some secret sauce. However, since the market is made of 100% of the people who transact in it, the presence of technical traders means that there must be some price movements as a result of these trends. One ideal would be to find a strategy that beats other technical analysts to the punch. A lot of people use the 12/26 MACD, but maybe the 11/25 MACD would help you eat their lunch before they entered the market...

It is all very tricky and takes a lot of discipline to avoid fooling yourself and getting into real trouble. Also, people need to make sure to focus on both sides of the trade - you only book the profit once you've actually sold the position (most people seem to be much more focused on entries than exits)

If anyone is interested in this kind of software but aren't so into learning a new language (perhaps less applicable to this group), you should check out QuantRunner Software http://www.quantrunner.com. In full disclosure, I am the CEO of the company, but we really focus on making this kind of back testing analysis easier for people to do without programming experience. We also have some novel tools for helping people actually improve their strategies, rather than simply iterate tests. If I had the true answers on the best strategies, I probably wouldn't run a software company. All we hope for is to make it easy enough for people to do their homework without having to worry about coding errors as well as poor strategies.

[steveplace]

Ideally, yes.

In practice, no. Trading a couple lots of /es futures will not do much to the market. More like pissing in the ocean. In illiquid markets you can move price, but not the stuff that you would want to blackbox anyways.

[mattmaroon]

It can. If someone has a large limit order and your small order hits their trigger price, whereas the next person might have done the opposite and moved away from it, you could have caused a huge change in the direction of the market. Or the same is true of a cascading series of smaller limit orders, etc.

And since making money is presumably based on volume, if you did small trades, you'd end up doing a lot of them to make it worth your while. But in almost all forms of betting, it's better to make fewer, better wagers.

[steveplace]

I don't know if you've traded /es futures, but that's nearly impossible to do. There's a couple different markets that put in bid/ask, and normally you'll just see the closest spread, but there's usually underlying bids as well. That's why it's so liquid. I'm looking at premarket futures right now and the qty of contracts offered at the bid/ask ranges from 10-100. That means to take the bid out you'd have to have a margin of around 100k.

And since making money is presumably based on volume, if you did small trades, you'd end up doing a lot of them to make it worth your while.

Not true with futures. They are risky, but you can get some pretty nice returns, especially in the volatility that we're having right now. On a single contract you can expect to pull 1k-2k, and that's on one trade per day. In terms of R, you're looking at 10R-20R per trade if you're experienced enough (>2 years).

[bayes]

The trouble with fewer better wagers is that wagers - even very good value ones - can lose. So if you're only making a few of them there's a significant probability of losing overall. Your expected value may be high, but your standard deviation is even higher.

I'd much rather have 10,000 independent bets each with a 0.5% edge than 10 independent bets each with a 20% edge.

[steveplace]

I'd much rather have 10,000 independent bets each with a 0.5% edge than 10 independent bets each with a 20% edge.

Not with trading, you want to cut losses quick and let winners run. The model behind Long Term Capital Management, as well as a ton of quant firms that went under this past Q, was the take-a-bunch-of-trades-for-small-profit... and it works until you get a six sigma event (see 2008).

[mixmax]

Yes, in that respect it's like physics - by measuring you are altering the experiment.

The volume on the day the reports came out was quite heavy, but if you had bought a substantial amount of stock it would certainly alter the market as you say. The point is that I'm pretty certain there are holes that will allow you to look at historic data and make statistically good buys - but you have to look where noone else is looking. Like the correlation of Nokia stock to the Finnish weather. Some of these will be large enough that you can make money, even though you alter the experiment.

On an unrelated note - I just read through your blog and found it very insightful :-)

[mattmaroon]

There's so much money to be made in the stock market (most of the money on Earth) and so many people searching for that data with so much resources (including math PhDs by the hundreds, some of whom I know) that looking somewhere nobody else is is virtually impossible. You're much better off focusing on your career and buying ETFs.

And thanks!

[kingkongrevenge]

forex dwarfs global equity markets.

[volida]

the first rule of trading is that you join the market not try to alter it.