You are referring to the critical period [1] of (second) language acquisition, which is generally thought to end with the onset of puberty [2].
Neurodevelopmentally, this period coincides with extensive synaptic and dendritic pruning and increased myelination (of axons) [3], which result in the loss of some connections and the strengthening and acceleration of others. Cell loss is not thought to be a major driver of brain maturation, nor is it thought to occur more frequently during this time window.