Since the introduction of effective anti-retroviral therapy, early diagnosis and treatment of HIV have become increasingly important from individual and public health perspectives. People who are diagnosed with a CD4 count below 350 cells/µL blood are today considered to be "late" diagnoses. In an effort to understand the reasons for late diagnosis, we conducted in-depth interviews (n = 14) in Amsterdam, the Netherlands. Two main factors were identified: psychosocial factors and health-system factors. Psychosocial factors relate to people's personal relationship with health professionals, low risk perception, fear related to the outcome of testing, and trauma from observed past experiences of living with HIV. Health-system factors relate to institutional barriers and missed opportunities during client-provider interactions. We conclude that in order to mitigate late diagnosis, the social and institutional context within which HIV testing is conducted should be addressed.