| Most "deep" networks are organized into layers and information flows in a particular direction although it doesn't have to be that way. Hinton wasn't saying we shouldn't have layers but that we should train the layers together rather than as black boxes that work in isolation. Also, when people talk about solving problems they talk about layers, layers play a big role in the conceptual models people have for how they do tasks even if they don't really do them that way. For instance in that ambiguous sentence somebody might say it hinges on whether or not you think "bite" is a verb or a noun. (Every concept in linguistics is suspect, if only because linguistics has proven to have little value for developing systems that understand language. For instance I'd say a "word" doesn't exist because there are subword objects that depend like a word "non-" and phrases that behave like a word (e.g. "dog bite" fills the same slot as "bite")) Another ambiguous example is this notorious picture https://www.livescience.com/63645-optical-illusion-young-old... which most people experience as "flapping" between two states. Since you only see one at a time there is some kind of inhibition between the two states. Who knows how people really see things, but if I'm going to talk about features I'm going to say that one part is the nose of one of the ladies or the chin of the other lady. Deep networks as we know it have nothing like that. |