[blackeyeblitzar]

Is that really true? If anything OpenAI was dependent on the transformers paper from Google from Ashish Vaswani and others. LeCun has been criticizing LLM architectures for a long time and has been wrong about them for a long time.

[mv4]

That was my impression too. He is considered the inventor of CNN back in 1998. Is there anything more recent that's meaningful?

[tedivm]

I was more referring to this paper from 2015:

https://scholar.google.com/citations?view_op=view_citation&h...

Basically all LLM can trace their origin back to that paper.

This was just a single example though. The whole point is that people build on the work from the past, and that this is normal.

[esafak]

That's just an overview for paper for those new to the field. The transformer architecture has a better claim to being the origin of LLMs.

[mv4]

Thank you for sharing this.

[blackeyeblitzar]

Personally, I have not seen anything from him that is meaningful. OpenAI and Anthropic (itself started by former OpenAI people) of course have built their models without LeCun’s contributions. And for a few years now, LeCun has been giving the same talk anywhere he makes appearances, saying that large language models are a dead end and that other approaches like his JEPA architecture are the future. Meanwhile current LLM architecture has continued to evolve and become very useful. As for the misuse of the term “open source”, I think that really began once he was at Meta, and is a way to use his fame to market Llama and help Meta not look irrelevant.

[tedivm]

They literally cited LeCun in their GPT papers.

[amelius]

By the way, as someone who once did classical image recognition using convolutions, I can't say I was very impressed by the CNN approach, especially since their implementation didn't even use FFTs for efficiency.