The COVID-19 pandemic prompted many countries to issue far-reaching policy measures that may have led to increased substance use. Higher education students may have been disproportionally affected due to the rearrangement of educational life and their susceptibility to psychosocial distress and substance use. The current study examined associations between pandemic-related stressors, psychosocial distress, and self-reported alcohol, tobacco, and cannabis use before and during the first wave of the pandemic. Data were collected in Belgium as part of the COVID-19 International Student Well-being Study (C19 ISWS) and analyzed using multinomial logistic regression analyses. The sample contained 18,346 higher education students aged 17 to 24 (75% women). Overall use of alcohol, tobacco, and cannabis as well as binge drinking decreased during the pandemic, perhaps due to limited social gatherings. Moving back to the parental home was associated with decreased substance use, while depressive symptoms were associated with increased substance use. Perceived threat and academic stress were associated with increased binge drinking among heavy bingers and increased tobacco use. Decreases among students who moved back to their parental home may be explained by increased informal social control. Increased substance use was associated with a number of stressors and psychosocial distress, which suggests that some students may have been self-medicating to manage their mental health amidst the pandemic. Public health policy concerning substance use may prove to be less effective if not tailored to particular subgroups within the student population.