[disgruntledphd2]

The main idea when you become a quant is that a computer is less prone to pitfalls than a human.

Computers and humans are prone to different pitfalls. Humans have far too many biases to count - see for example, the works of Kahneman and Tversky and most of social psychology.

Computers, on the other hand come with a whole host of different problems (perhaps because they're made by humans). The essential advantage a human has is the eye, which is extremely well adapted to picking out patterns. That, and an ability to go beyond the model or completely change it. This is something that computers have difficulty with (unless of course they're programmed for it - i think genetic algorithms claim to do this, but I'm not particularly knowledgeable about those).

Nonetheless, i agree with the thesis that this kind of analysis will invade the rest of the social sciences. In fact, that's one of the reasons I learned to program.

[mdda]

One of the things about financial markets is that large numbers of people are attempting to spot patterns, and eek out a profit from the patterns repeating. People are very good at this - which means that, over time, the number of profitable patterns reduces. Thus, to human eyes, market behaviour becomes noise : just like zip compression reduces a bytestream to being essentially white noise by taking out repetitive sequences.

Computers are just the next step, crunching out the patterns until they are unexploitable (below the threshold of trading costs).

The end result is that markets are a random walk - unless you are at the bleeding edge with faster machines, better latency, lower transaction costs, etc.

Of course, an alternative to this is to do true bottom-up analysis, or invest in illiquid companies (like VCs do).

[marchdown]

This is a strong point, and the one rarely acknowledged at that. Many people stop at pointing out that efficient markets cannot be gamed, and almost all exploitable inefficiencies have been ironed out already. In correctly observing that most investment activities are fueled by greed and human biases, they incorrectly extend this to trading illiquid goods, which still have a lot of low-hanging fruit to pick.

[radikalus]

One counterpoint is that at the bleeding edge of low-latency, pattern analysis and sig-int become once again extremely meaningful.

Can you execute your strategy faster than your opponents if you go to a slightly more aggressive (less arbitrage-y) signal?

Should you? (Why bother if you're faster?)

For what situations is it worth "thinking longer"? Some straight arbs require speed beyond what you can do if you want to use your HOT "smart" model.

If you work outwards from the fastest "stupidest" trades, there's a vast array of strategies/opportunities that intersect ML/AI, hardware design, network optimization, and so forth -- I do agree that if you're looking at bad, inaccurately sampled tick data, the opportunities aren't really there anymore. (Because there's increasingly more players correcting relative value mispricings)

[disgruntledphd2]

I think the real difficulty for prediction of the stock market is that your theory will be understood by others who will alter their behaviour in response to their theory, in essence invalidating it. Its actually a problem across the social sciences, witness the inflitration and acceptance of Freud and Jung's ideas which are no so much part of the culture that theories cannot be built on them (they were always a little suspect anyway though).

[SkyMarshal]

> The essential advantage a human has is the eye, which is extremely well adapted to picking out patterns.

I'm not sure that's an advantage. The human eye is so good at discerning patterns that it sees them even where they do not exist. Witness: technical analysis, Eliot Wave Theory, etc.

Quants use math to provide a more rigorous framework for eliminating hocus pocus like that, though they have been known to make less than rigorous assumptions from time to time (good ones like Paul Wilmott have been particularly prescient in calling out that tendancy).

> Nonetheless, i agree with the thesis that this kind of analysis will invade the rest of the social sciences. In fact, that's one of the reasons I learned to program.

Agreed. There's still anachronistic cruft that needs to be exorcised from the field, for example the notion of 'utility' that economists use to evaluate the psychology of decision making (good discussion about that on HN recently, forgot where). CS + X, for (almost) all X, is where the world is heading.

[disgruntledphd2]

I really wouldn't call picking out patterns by eye as hocus pocus, I see where you're coming from, but I find (in my own work) that a good graph can help me to understand a model. Again, this does play into the biases of humans, but given enough awareness of these, the intersection of algorithms to find automatic patterns and the ability of the human eye and mind to discern meaning and give interpretation to these patterns is a very powerful combination.