Most of the literature suggests a positive relationship between immigrant concentration and anti-immigrant sentiments. The main goal of this study is to investigate the impact of both perceived and actual size of migrant populations on anti-immigrant sentiments. A representative survey of inhabitants of local communities in the Flemish region of Belgium shows a strong tendency to overestimate the presence of non-nationals. The survey allows us to conclude that respondents living in ethnically diverse communities do not have more negative attitudes towards immigrants. Individuals who perceive more immigrants to be present in their communities are more hostile even after controlling for reported contact with members of immigrant groups. We can therefore conclude that the perceived size of the immigrant group has a stronger impact on anti-immigrant sentiments than the actual presence of ethnic minority groups.