[SirIsaac]

Dreyfus was a very intelligent man. He knew the difference between representational and non-representational systems. As Yann LeCun has said many times, deep learning is the learning of representations. That's all anyone needs to know in order to understand Dreyfus's thesis. The brain does not need a prior representation of a bicycle to perceive a bicycle. A DNN would be blind to a bicycle without a prior representation. Did you read the article?

[Firadeoclus]

What exactly does it mean to "perceive a bicycle"? Noticing shapes and colours and recognising them as a distinct object? Recognising an obstacle? Noticing qualities like smoothness and straightness and associating the concept "man-made"? Being able to explain its purpose? Predicting how it might move, if ridden by a person?

[SirIsaac]

Yes, pretty much. In my opinion, perception is generalization. To perceive a bicycle is to perceive many types of qualities or properties about it that can also be applied to a potentially infinite number of other objects. The brain can perform this generalization instantly without having stored previous representations (bicycle patterns) in memory.

A great example of non-representational intelligence is the honeybee's brain. It has less than 1 million neurons but it can handle zillions of patterns/objects in its 3D environment. It would be impossible for it to store all those zillions of patterns in its tiny brain. It uses the world itself as its own model.

For these reasons, deep learning is irrelevant to AGI.