BACKGROUND: Tobacco, alcohol, and cannabis use are prevalent behaviors that pose considerable health risks. Genetic vulnerability and characteristics of the neighborhood of residence form important risk factors for substance use. Possibly, these factors do not act in isolation. This study tested the interaction between neighborhood characteristics and genetic risk (gene-environment interaction, GxE) and the association between these classes of risk factors (gene-environment correlation, rGE) in substance use. METHODS: Two polygenic scores (PGS) each (based on different discovery datasets) were created for smoking initiation, cigarettes per day, and glasses of alcohol per week based on summary statistics of different genome-wide association studies (GWAS). For cannabis initiation one PGS was created. These PGS were used to predict their respective phenotype in a large population-based sample from the Netherlands Twin Register (N = 6,567). Neighborhood characteristics as retrieved from governmental registration systems were factor analyzed and resulting measures of socioeconomic status (SES) and metropolitanism were used as predictors. RESULTS: There were (small) main effects of neighborhood characteristics and PGS on substance use. One of the 14 tested GxE effects was significant, such that the PGS was more strongly associated with alcohol use in individuals with high SES. This was effect was only significant for one out of two PGS. There were weak indications of rGE, mainly with age and cohort covariates. CONCLUSION: We conclude that both genetic and neighborhood-level factors are predictors for substance use. More research is needed to establish the robustness of the findings on the interplay between these factors.