[srean]

Simpson's Paradox is one of the many phenomena that shows how different applied ML is from regular software engineering. Another one is feedback loops between decomposed subproblems.

In ML encapsulation, shielding away of inner details often does not work. One needs to know what is happening on the other side of the abstraction boundary. This is a problem for managers and PM coning to ML from a purely software engineering background. They are used to encapsulation and decomposition serving them well and they expect the same.

[brians]

You’re right about ML. But you’re mistaken about software engineering—though in good company with most software engineers.

Of denotation, cache access, confidentiality, authentication, integrity, non-repudiability, performance, thread safety, memory overhead—only denotation and some parts of memory overhead allow composition of abstractions.

[jonahx]

Would you mind elaborating on this?

[heavenlyblue]

>> In ML encapsulation, shielding away of inner details often does not work.

I call bs on this. It’s just that we haven’t yet invented a consistent type theory on top of ML.

[apathy]

“Just”

This would be like saying “it’s just that we haven’t proven P!=NP” in CS. Best of luck.

Meanwhile applied people will deal with the problem by model diagnostics and sensitivity analysis as has been done for decades. I can’t wait for the next AI winter to come. So tired of this handwaving by people who don’t seem to have practical experience.

[srean]

Whoah! You have quite a treasure trove in your favorites. The possibility of getting some work done vanished as soon as I found that.

[apathy]

Always happy to be a bad influence.

[heavenlyblue]

Yeah, but you just reduced the parent comment to ML being the same as CS. And the parent is saying the opposite: that they differ from each other.

So meanwhile, speaking of applied knowledge... I believe you didn't even read what you're replying to.