Puberty is characterized by major changes in hormone levels and structural changes in the brain. To what extent these changes are associated and to what extent genes or environmental influences drive such an association is not clear. We acquired circulating levels of luteinizing hormone, follicle stimulating hormone (FSH), estradiol and testosterone and magnetic resonance images of the brain from 190 twins at age 9 [9.2 (0.11) years; 99 females/91 males]. This protocol was repeated at age 12 [12.1 (0.26) years] in 125 of these children (59 females/66 males). Using voxel-based morphometry, we tested whether circulating hormone levels are associated with grey matter density in boys and girls in a longitudinal, genetically informative design. In girls, changes in FSH level between the age of 9 and 12 positively associated with changes in grey matter density in areas covering the left hippocampus, left (pre)frontal areas, right cerebellum, and left anterior cingulate and precuneus. This association was mainly driven by environmental factors unique to the individual (i.e. the non-shared environment). In 12-year-old girls, a higher level of circulating estradiol levels was associated with lower grey matter density in frontal and parietal areas. This association was driven by environmental factors shared among the members of a twin pair. These findings show a pattern of physical and brain development going hand in hand.