STUDY DESIGN: Reliability study. OBJECTIVES: The aim of this study was to evaluate the interrater-reliability of the interpretation of diagnostic ultrasound in patients with shoulder pain between physical therapists and radiologists. BACKGROUND: Although physical therapists in The Netherlands increasingly use diagnostic ultrasound in clinical practice, there is no evidence available on its reliability. METHODS: A cohort study included patients with shoulder pain from primary care physiotherapy. Patients followed the usual diagnostic pathway of which diagnostic ultrasound could be a part. Patients that received diagnostic ultrasound also visited a radiologist within one week for a second one. Patients and radiologists were blinded for the diagnostic ultrasound diagnosis of the physical therapists. Agreement was assessed using Cohen's kappa statistics. Subgroup analysis was performed on education and experience. RESULTS: A total of 65 patients were enrolled and 13 physical therapists and 9 radiologists performed diagnostic ultrasound. We found substantial agreement (0.63 K) between physical therapists and radiologists on the assessment of full thickness tears. The overall kappa of all four diagnostic categories was 0.36, indicating fair agreement. The more experienced and highly trained physical therapists showed moderate agreement (0.43 K) compared to only slight agreement (0.17 and 0.09 K) from the less experienced and trained physical therapists with radiologists. CONCLUSION: The reliability between physical therapists and radiologist on diagnostic ultrasound of shoulder patients in primary care is borderline substantial (Kappa = 0.63) for full thickness tears only. This level of reliability is relatively low when compared with the high reliability between radiologists. More experience and training of physical therapists may increase the reliability of diagnostic ultrasound.