[Legend2440]

You're missing a step. The perturbations are not fully random. The LLM also looks at the result and tries to do credit assignment to determine what changes to try in the next round.