[superpermutat0r]

Linear regression can be the reduction form of many sophisticated methods. For example, HMM or CRF like inference can be easily done with linear regression.

Hierarchical models with several layers of symbolic inference can be done with linear regression.

Reinforcement learning too.

You can reduce almost anything to a binary classifier and implement it in practice to work well. The reduction is tricky but performance is not.

[hogFeast]

As I explained, he said it was none of this.

[superpermutat0r]

I do not see where you explained it. Doing cost sensitive linear regression (which is pretty trivial to implement) allows you to approximate Markov models, any hierarchical model, and all sorts of other stuff (like minimizing different cost functions, quantile regression etc.) All achievable with the same linear regression algorithm and additional data modifications.

Maybe sometimes you need to change the weight update rule, but that's it.

[hogFeast]

It is funny to watch you struggle with this (maybe this is why you don't see). He specifically said that wasn't what they were doing (and said it was often simple linear regression with one or two variables), they used no complicated techniques (he even pointed out that people assume that must be true...but it isn't), and that what they did was a combination of good analytical work/hiring. Btw, this is also quite obvious from reading Zuckerman's book.

I understand why almost no-one gets it. And that is why firms like RenTech are able to print money. I actually do work like this in a similar field, and everyone assumes you have some kind of secret algo that is the product of some unpublished, very complex work (these days, usually deep/reinforcement learning). But the reality is doing the simple stuff well and using that to build a deep understanding of the data. I suppose that is less fun for researchers who want to publish flashy stuff about deep learning and write lots of equations on whiteboards...but I prefer the money.

[superpermutat0r]

I do not even know about RenTech. I was just pointing out that there are several tiny hacks one can do to significantly expand capabilities of linear regression. Just like you can do polynomial regression with some data modification, you can do practically anything I mentioned above. It's very simple, modifications you can use in minutes.