Background: The impact of the COVID-19 pandemic among people with physical disabilities might differ between countries due to differences in implemented measures and infection rates. This study aimed to understand the impact of the pandemic on physical activity (PA) and social isolation among adults with physical disabilities in Canada and the Netherlands, and examine associations between PA and social isolation. Methods: Secondary data from two studies were used: the Canadian COVID-19 Disability Survey (n = 353) and the Dutch Rehabilitation, Sports and Active lifestyle (ReSpAct) 2.0 study (n = 445). Self-reported PA was measured using IPAQ-SF and Adapted-SQUASH. Social isolation was measured using the PROMIS Social Isolation. Descriptive and regression analyses were performed. Results: Canadian participants spent on average 163 min (Median = 0; IQR = 120) on moderate-to-vigorous PA per week and Dutch participants 934 min (Median = 600; IQR = 1125). In Canada, 64% reported to have become less physically active since the pandemic compared to 37% of Dutch participants. In both samples, no clinically relevant associations were found between PA and social isolation. Conclusions: The findings emphasize the negative impact of the pandemic on PA and social isolation in adults with physical disabilities in Canada and the Netherlands. Future research is needed to better understand if and how PA can be used to reduce social isolation in people with disabilities. This study illustrates how cross-country collaborations and exchange provide opportunities to inspire and learn from initiatives and programs in other countries and may help to improve PA support among people with disabilities during and after the pandemic.