This study examines the relationship between residential segregation and social trust of immigrants and natives in the Netherlands. Building on previous studies that have found evidence for a negative segregation-trust link, we present a nuanced narrative by (i) distinguishing between an ethnic minority and majority perspective, (ii) elaborating theoretical foundations on the moderating role of individual exposure in the form of ethnic minority concentration in the neighborhood, and (iii) taking income segregation into account. In addition to the refined theoretical framework, our study employs a rigorous empirical approach. Using two waves (2009 and 2013) of the Netherlands Longitudinal Lifecourse Study—a geocoded panel study with an oversampling of Moroccan and Turkish immigrants—we are able to study the influence of (changes in) municipality-level segregation patterns for both natives and immigrants, and consider the roles of both neighborhood ethnic minority concentration, as well as income-based segregation. Results from four-level multilevel models show that ethnic segregation is negatively related to the social trust of immigrants. At the same time, this negative relationship is particularly strong in neighborhoods with a low level of minority population concentration, which provides support for the so-called integration paradox where negative intergroup interactions reduce social trust. For respondents of Dutch origin, we find no evidence that their social trust is sensitive to ethnic segregation or that this relationship is conditional on minority concentration at the neighborhood level.