Objective: To determine the accuracy of computed tomography (CT)-based staging in selecting high-risk colon cancer patients who would benefit from neoadjuvant chemotherapy while avoiding overtreatment. Methods: Data of adult patients diagnosed with non-metastatic primary colon cancer in 2005–2020, who underwent surgical resection without neoadjuvant chemotherapy, were retrospectively collected from the Netherlands Cancer Registry. Agreement between clinical and pathological evaluation for each T and N stage was calculated. Sensitivity and specificity analyses were conducted to predict T3-T4 and N1-N2 stages, with histopathology as the reference standard. Results: Data from 44,471 patients (median age, 71 years, 50% female) were evaluated. We included 38,915 patients with complete T stage and 39,565 patients with complete N stage for analyses. The overall clinical-pathological agreement for T stage was 59% and for N stage 57%. The sensitivity and specificity of CT to detect T3-T4 tumours were 80% (95% confidence interval (CI): 0.79, 0.80) and 76% (95% CI: 0.75, 0.77), respectively, with a positive predictive value (PPV) of 92% (95% CI: 0.92, 0.92). The sensitivity and specificity of CT to detect N1-N2 category were 62% (95% CI: 0.61, 0.63) and 70% (95% CI: 0.69, 0.71), respectively, with PPV 60% (95% CI: 0.59, 0.60). Conclusion: CT-based staging shows limited accuracy in selecting colon cancer patients who would benefit from neoadjuvant therapy without risking overtreatment. Detection of lymph node metastases with CT remains unreliable.