Background: This study aims to (1) investigate the prevalence of anxiety, depression and severe fear of cancer recurrence or progression in gastrointestinal stromal tumor (GIST) patients treated in a curative or palliative setting, (2) compare their prevalence with a norm population, (3) identify factors associated with anxiety, depression and severe fear, and (4) study the impact of these psychological symptoms on health-related quality of life (HRQoL). Methods: In a cross-sectional study, GIST patients completed the Hospital Anxiety and Depression Scale, Cancer Worry Scale, and EORTC QLQ-C30. Results: Of the 328 patients, 15% reported anxiety, 13% depression, and 43% had severe fear. Anxiety and depression levels were comparable between the norm population and patients in the curative setting, but significantly higher for patients in the palliative setting. Having other psychological symptoms was associated with anxiety, while current TKI treatment and anxiety were associated with depression. Severe fear was associated with age, female sex, palliative treatment setting, anxiety, and GIST-related concerns. Conclusion: GIST patients treated in a palliative setting are more prone to experience psychological symptoms, which can significantly impair their HRQoL. These symptoms deserve more attention in clinical practice, in which regular screening can be helpful, and appropriate interventions should be offered.