Background Referral to specialized palliative care services (SPCS) occurs often late in the illness trajectory but may differ across cancer types. We examined differences between cancer types in the use and timing of referral to specialized palliative care services (SPCS) and in the reasons for non-referral. Methods We conducted a population-based mortality follow-back survey among physicians who certified a representative sample of deaths in Flanders, Belgium. We focused only on sampled death cases of cancer (n = 2392). The questionnaire asked about the use of the existing types of SPCS and the timing of referral to these services. Results Response rate was 58% (1394/2392). Patients who died from breast, respiratory, head and neck, genitourinary or gastrointestinal cancer had higher chances of using SPCS compared to hematologic cancer patients. The most prevalent reason for non-referral was that regular care sufficiently addressed palliative and supportive care needs (51%). This differed significantly between cancer types ranging from 77,8% for breast cancer and 42.1% for hematologic cancer. A second prevalent reason for not using SPCS was that it was not meaningful (enough) (23.9%), particularly for hematologic malignancies (35,1%) and only in 5.3% for breast cancer. Conclusion Differences in referral across different types of cancer were found. Referral is more often delayed or not initiated for patients with hematologic cancer, possibly due to differences in illness trajectory. An influencing reason is that physicians perceive palliative care as not meaningful or not meaningful enough for these patients which may be linked to the uncertainty in the disease trajectory of hematologic malignancies.