It starts before then. Babies only a few days old show a preference for Duplo or dolls depending on their biological sex.
I can't find the paper, though.