The interpretation of size terms involves constructing contextually-relevant reference points by combining visual cues with knowledge of typical object sizes. This study aims to establish at what age children learn to integrate these two sources of information in the interpretation process and tests comprehension of the Dutch adjectives groot ‘big’ and klein ‘small’ by two- to seven-year-old children. The results demonstrate that there is a gradual increase in the ability to inhibit visual cues and to use world knowledge for interpreting size terms. Two- and three-year-old children only used the extremes of the perceptual range as reference points. From age four onwards children, like adults, used a cut-off point in the mid-zone of a series. From age five on, children were able to integrate world knowledge and perceptual context. Although seven-year-olds could make subtle distinctions between sizes of various object classes, their performance on incongruent items was not yet adult-like.