Background Patient satisfaction is one of the relevant indicators of quality of care; however, measuring patient satisfaction had been criticised. A major criticism is that many instruments are not reliable and/or valid. The instruments should have enough discriminative power for benchmarking of the results. Objective To develop a “core questionnaire for the assessment of patient satisfaction in academic hospitals” (COPS) that is reliable and appropriate for benchmarking patient satisfaction results. Research design The development of COPS, the testing of its psychometric quality and its use in eight Dutch academic hospitals in three national comparative studies in 2003, 2005 and 2007 are described in this study. Results were reported only if they were significant (p<0.05) and relevant (also Cohen d>0.2). Results The questionnaire was returned in 2003 by 40 678 patients (77 450 sent, 53%) and by 40 248 patients (75 423 sent, 53%) in 2005. In 2007, the questionnaire was returned by 45 834 patients (87 137, 53%). The six dimensions have good Cronbach α's, varying from 0.79 to 0.88.The results of every item were reported to the individual hospital. A benchmark overview showed the overall comparison of all specialties of the eight hospitals for the clinic and outpatient departments. The 2007 measurement showed relevant differences in satisfaction on two dimensions in the clinical setting. Conclusions COPS is shown to be a feasible and reliable instrument to measure the satisfaction of patients in Dutch academic hospitals. It allows comparison of hospitals and gives benchmark information on a hospital as well as data on specialty levels and previous measurements, including best practices.