[mlthoughts2018]

I'm sorry but I simply don't agree about the politeness comment. As linked from a Quora post that goes into, this was one of Pearl's original statements about the disagreement (link to the original at the UCLA site appears to have been taken down) [0]:

> "I therefore invite my colleagues... to familiarize themselves with the miracles of do-calculus. Take any causal problem for which you know the answer in advance, submit it for analysis through the do-calculus and marvel with us at the power of the calculus to deliver the correct result in just 3–4 lines of derivation. Alternatively, if we cannot agree on the correct answer, let us simulate it on a computer, using a well specified data-generating model, then marvel at the way do-calculus, given only the graph, is able to predict the effects of (simulated) interventions. I am confident that after such experience all hesitations will turn into endorsements. BTW, I have offered this exercise repeatedly to colleagues from the potential outcome camp, and the response was uniform: “we do not work on toy problems, we work on real-life problems.” Perhaps this note would entice them to join us, mortals, and try a small problem once, just for sport."

This is absolutely the cheeky spirit of one-upsmanship I am talking about. The offers are always framed in terms of "look how causal inference supersedes everything," which is not a charitable take on approaches from others, especially in historical applied ML, that might have already developed some of the same underlying ideas.

[0]: https://www.quora.com/Why-is-there-a-dispute-between-Judea-P...

[closed]

I don't know. The issue he is addressing in your quote is that people often leverage criticisms of his approach that are just verbal statements. Pearl wants people to use data-generating models to make their concerns explicit.

The link you used explains the situation pretty well. If anything Pearl's regular acknowledgement of graphical models seems to be an indication that he is mindful of at least one very common approach in current ML.

[fjsolwmv]

Isn't it incumbent on Pearl or someone on the do-calculus school to run an experiment to show it performs better than popular ML systems?

It's a beautiful theory, but it hearkens back to the symbolic AI era that has had limited effectiveness.

[closed]

In theory, yes. However, I think in practice addressing the concerns of critics is often out of Pearl's hands.

Until they supply a "ground truth" or data generating model, he has a dilemma:

* if he doesn't create a data generating model, then arguments for / against his approach will be specious.

* if he creates a data generating model, they can claim it doesn't reflect reality.

In the case of Judea Pearl and Andy Gelman, it seems like the point of contention is much broader than the do-calculus. Andy Gelman does not seem to be a fan of structural equation modeling / similar graphical models.

[mlthoughts2018]

How is it out of Pearl’s hands? Also, Gelman & Rubin already did look into Pearl’s models, and even agreed that for some toy model examples, the technique works as intended, but that there are serious how-things-work-in-practice reasons why Pearl’s models are unlikely to be mathematically appropriate for some real world use cases.

It’s really a fair response from them to Pearl, especially when the whole time Pearl is presenting it like causal inference is a miracle cure-all.

All I am seeing in your comments is hand waving attempts to shift the burden of proof onto the group of practitioners who already looked into this stuff and weren’t convinced!

So why does it being incumbent on Pearl or on another causal inference practitioner to demonstrate it scaling up to a more complicated in-practice problem still get qualified with an “in theory” from you? Why isn’t it resoundingly obvious by this point that the burden of proof lies with Pearl, and that people would be happy to hear if he can use these models for large-scale, practical use cases, but they (rightfully) don’t see a reason (even after looking into the models) to spend their own time doing it?

[stevenhuang]

As an outsider, that reads as quite polite, especially given the context of his quote as what seems to be a request fallen on deaf ears.

I read the Quora link. It seems most people are supportive of Pearl's ideas. Why not try his proposed thought experiment and see?

[mlthoughts2018]

Many people have already and continue to try his experiment, and indeed work hard on scaling up the problems that his method is applied to.

That’s part of the problem. It was offered up as a “miracle” that supersedes and fully, logically subsumes the hard fought and real-world tested methods of others, who rightfully weren’t going to advocate the use of some other, unproven thing claimed to be a cure-all, yet they still did engage heavily with it and worked through the math and agreed with derivations for simple models under various collections of assumptions.

I’m still not seeing anything convincing. Pearl’s tone is not polite or even quirky, and strays badly from any measure of humility or collaborative spirit of inquiry. Really. I mean, sure, he’s not coming out dropping f-bombs, but that’s utterly not the point. Trying to justify it as if it’s a diplomatic and earnest request is silly. “Hey, your career’s worth of work is totally wrong. Here look, my work completely supersedes yours because I worked through some canonical, highly academic cases. But I’m being fair & balanced about it, I swear!”

And then when folks did look into it and still felt unconvinced, “I guess you’re just dogmatic & closed minded. I offered to look into it with you, but you ignored me.”