Background: Doing regular physical activity has positive effects on health. Several environmental factors are identified as important correlates of physical activity. However, there seems to be a difference between perceived and objective measures of the environment. This study examines the influence of physical activity, neighbourhood walkability, and socio-demographic characteristics on the correspondence between self-reported and objectively measured walking time to urban destinations of adults in the city of Ghent (Belgium). Methods: Previously collected survey data was used from 1164 respondents in the city of Ghent who reported walking times to various closest destinations in the neighbourhood of residence. These were compared with corresponding walking times that were objectively measured through geographical information systems. Physical activity was recorded over a 7-day period using accelerometers. Neighbourhood walkability was assessed on the basis of residential density, connectivity, and land-use mix. Results: We observed a relatively poor agreement between objective and perceived walking times. Stronger agreements were noted amongst the most physically active group, while low-level walkers tended to overestimate walking time. Surprisingly, however, people residing in a low-walkable neighbourhood underestimated walking times more frequently relative to those in high-walkable neighbourhoods. Conclusions: Researchers investigating the influence of environmental attributes on physical activity behavior should thus be cautious when using only self-reported environmental data, since these are a priori influenced by physical activity levels and various socio-demographic factors.