Newborn babies can hardly do anything because their parameters have not been set; but their hyperparameters (things like, to use NN analogies: activation function, learning rate, etc.) are baked in.
Right. Evolution picked the hyperparameters, and experiential learning picks the parameters. (To first order; certainly there are some instinctual behaviors programmed by evolution and I wouldn't be surprised if some hyperparameters are influenced by experience as well.)