[YeGoblynQueenne]

From the paper:

>> In this paper, we present an attempt at an architecture which operates on an explicit higher-level semantic representation, which we name a “concept”.

I wonder if the many authors of the paper know that what they call "concept" is what all of machine learning and AI has also called a "concept" for many decades, and not a new thing that they have just named from scratch.

For instance, classes of "concepts" are the target of learning in Leslie Valiant's "A Theory of the Learnable", the paper that introduced Probably Approximately Correct Learning (PAC-Learning). Quoting from its abstract:

  ABSTRACT: Humans appear to be able to learn new
  concepts without needing to be programmed explicitly in
  any conventional sense. In this paper we regard learning as
  the phenomenon of knowledge acquisition in the absence of
  explicit programming. We give a precise methodology for
  studying this phenomenon from a computational viewpoint.
  It consists of choosing an appropriate information gathering
  mechanism, the learning protocol, and exploring the class of
  concepts that can be learned using it in a reasonable
  (polynomial) number of steps. Although inherent algorithmic
  complexity appears to set serious limits to the range of
  concepts that can be learned, we show that there are some
  important nontrivial classes of propositional concepts that
  can be learned in a realistic sense

From: https://web.mit.edu/6.435/www/Valiant84.pdf

Or take this Introduction to Chapter 2 in Tom Mitchell's "Machine Learning" (the original ML textbook, published 1997):

  This chapter considers concept learning: acquiring the definition of 
  a general category given a sample of positive and negative training 
  examples of the category.

From: https://www.cs.cmu.edu/~tom/mlbook.html (clink link in "the book").

I mean I really wonder some times what is going on here. There's been decades of research in AI and machine learning but recently papers look like their authors have landed in an undiscovered country and are having to invent everything from scratch. That's not good. There are pitfalls that all the previous generations have explored thoroughly by falling in them time and again. Those who don't remember those lessons will have to find that out the hard way.

[mdp2021]

I am not sure that fits the point, YGQ:

it seems to me the concept of «concept» in the paper is "the embedding vector we get in systems like SONAR (which we could use to generalize ordered sets of tokens into more complex ideas)". That's pretty specific, only marginally related to past handling as mentioned.

[YeGoblynQueenne]

That's only the representation of a concept. Different systems and different approaches will have different representations but that doesn't change the fact of what is being represented.

[mdp2021]

But if the issue is about "research in AI has had to deal with the concept of "concept" since the inception" (and of course it had to), the contribution in this paper is to try an operational implementation that could bear fruit and possibly fix architectural shortcomings of the mainstream effort.

(It is not separate from the context of LLMs.)

[YeGoblynQueenne]

Right, but there's been many operationalisations before. That's what's not new. Tome Mitchell's textbook has plenty of examples. Basically all of machine learning is about learning concepts- in practice as well as in theory. That's the whole point.