| From the comments on that post, written by LeCun: "'[...] Yann LeCun, [...] is on a mission to reposition himself, not just as a deep learning pioneer, but as that guy with new ideas about how to move past deep learning' First, I'm not 'repositioning myself'. My position paper is in the direct line of things I (and others) have thought about, talked about, and written about for years, if not decades. Gary has merely crashed the party. My position paper is not at all about 'moving past deep learning'. It's the opposite: using deep learning in new ways, with new DL architectures (JEPAs, latent variable models), and new learning paradigms (energy-based self-supervised learning). It's not at all about sticking symbol manipulation on top of DL as he suggests in vague terms. It's about seeing reasoning as latent-variable inference based on (hopefully gradient-based) optimization. Gary claims that my critiques of supervised learning, reinforcement learning, and LLMs (my 'ladders') are critiques of deep learning (his 'ladder'). But they are not. What's missing from SL, RL and LLM are SSL, predictive world models, joint-embedding (non generative) architectures, and latent-variable inference (my rockets). But deep learning is very much the foundation on which everything is built. In my piece, reasoning is the minimization of an objective with respect to latent variables. If Gary wants to call this 'symbol manipulation' and declare victory, fine. But it's merely a question of vocabulary. It certainly is very much unlike any proposal he has ever made, despite the extreme vagueness of those proposals." |