[magicalhippo]

> I have yet to see a real world production pipeline where GBDT provides enough improvement over Logistic Regression

Not my field at all, so "I know nooothing".

Are GBDT's very different from "plain" binary decision trees? I've seen the latter a lot in the context of particle experiments[1][2][3].

[1]: https://arxiv.org/abs/physics/0408124

[2]: http://cds.cern.ch/record/2289251/

[3]: https://arxiv.org/abs/2002.02534

[hogFeast]

Very simply: plain decision trees usually overfit to training data (and, therefore, perform very badly out of sample). So the important part isn't the tree but the boosting. How you go from an ensemble of weak learners to something that works.

And this boosting generalises to any learner. You can apply it to regression too. Again, the boosting part is really the key. The innovation isn't a new technique either, it is just the aggressive application of computing power to these problems.

[jncraton]

They are the same concept under the hood, but a GBDT is an ensemble model using a number of trees in tandem that are grown to improve the performance of the overall model.