Sedentary behavior has negative health effects. It is assumed that the walkability of the living environment is related to the amount of time spent on sedentary behavior in the residential setting. However, evidence on such a relation is still scarce, and results are contradictory. Therefore, we examined to what extent residential neighborhood walkability is associated with a variety of sedentary behaviors that frequently occur in the residential setting among adults. We carried out a cross-sectional survey using the domain-specific sedentary behavior questionnaire among adults in Breda, the Netherlands. Respondents’ walkability of the living environment was assessed objectively by means of road network buffers. We employed gamma generalized linear regressions to assess correlations between multiple sedentary behaviors and neighborhood walkability. We found no significant associations between residential neighborhood walkability and sedentary behavior levels. The lack of correlations was consistent across buffer sizes. Our models showed that adults with a higher education, a job, and a driver’s license spent significantly less time on sedentary behaviors. Our findings suggest that person-level characteristics should be targeted when developing intervention strategies to counteract sedentary time, rather than intervening in the walkability of the residential living environment.