|
I did my Ph.D. work on the cerebellum and proposed a theoretical model of reinforcement learning and can add some context for why this is a "big deal". First, it represents a fundamental shift in how the cerebellum learns. For a long time, it was thought that the cerebellum learned liked a perceptron neural network, i.e. that an error signal was computed and used to change the strength of synapses within the cerebellum to result in the correct output signal to guide motor control (and, we know now, cognitive control). In other words, the cerebellum was a supervised learning machine [1]. But how these error signals were actually computed using the neuronal circuitry was never made clear; most arguments centered around the microcircuitry of the inferior olive. In a perceptron, an error signal is the difference between the correct output and the predicted output. But how is the "correct output" supplied? As far as I know the first to propose the cerebellum learned by reinforcement learning was the famed cerebellum researcher Richard Thompson [2]. Unfortunately, the idea was only vaguely sketched out the field didn't take this very seriously and continued on with the general belief that the cerebellum learned by supervised learning. To me and my collaborator, Tadashi Yamazaki, it seemed a more natural signal that the nervous system could capably supply would be a graded reward signal. Moreover, this meant we could interpret the structure of the cerebellum within the theoretical frameworks of reinforcement learning such as the actor-critic framework [3,4]. This is the second reason these finding are a big deal: the paradigm shift to the cerebellum being a reinforcement learning machine, if correct, will be a boon for building better models of it. In the last few years there has been some impressive work done with reinforcement learning in artificial neural networks that could be applied to models of the cerebellum, especially within context of the brain at large. [1] Doya, Kenji "What are the computations of the cerebellum, the basal ganglia and the cerebral cortex" (1999)
[2] Thompson, Richard "The nature of reinforcement in cerebellar learning" (1998).
[3] Lennon, William "Towards more biologically plausible computational models of the cerebellum with emphasis on the molecular layer interneurons" (2015)
[4] Yamazaki and Lennon "Revisiting a theory of cerebellar cortex" (2019) |
If, like me, you wondered how a poor fruit has microcircuitry, well, I think they're actually talking about the Inferior Olivary Nucleus. Per Wikipedia:
> The inferior olivary nucleus (ION), is a structure found in the medulla oblongata underneath the superior olivary nucleus. In vertebrates, the ION is known to coordinate signals from the spinal cord to the cerebellum to regulate motor coordination and learning. These connections have been shown to be tightly associated, as degeneration of either the cerebellum or the ION results in degeneration of the other.
https://en.wikipedia.org/wiki/Inferior_olivary_nucleus