Background: In the Netherlands, the clinical benefit of systemic anti-cancer treatments (SACTs) is assessed by the Committee for the Evaluation of Oncological Agents (cieBOM). For non-curative SACTs, the assessment is based on the hazard ratio (HR) for progression-free survival and/or overall survival (OS), and the difference in median survival. We evaluated the impact of different thresholds for effectiveness by reassessing the clinical benefit of SACTs. Methods: We reassessed SACTs that were initially assessed by cieBOM between 2015 and 2017. Four scenarios were formulated: replacing an “OR” approach (initial assessment) by an “AND” approach (used in all scenarios), changing the HR threshold from < 0.70 (initial assessment) to < 0.60, changing the threshold for the difference in median survival from > 12 weeks (initial assessment) to > 16 weeks, and including thresholds for OS rates. The outcomes of these scenarios were compared to the outcomes of the initial assessment. Results: Reassessments were conducted for 41 treatments. Replacing the “OR” approach by an “AND” approach substantially decreased the number of positive assessments (from 33 to 22), predominantly affecting immunotherapies. This number further decreased (to 21 and 19, respectively) in case more restrictive thresholds for the HR and difference in median survival were used. Including thresholds for OS rates slightly mitigated the impact of applying an “AND” approach. Conclusions: The scenario-specific thresholds had a substantial impact; the number of negative assessments more than doubled. Since this was not limited to treatments with marginal survival benefits, understanding the potential challenges that may arise from applying more restrictive thresholds is essential.