Immigrants are often found to rate their health better than the native population does. It is, however, suggested that this healthy immigrant effect declines with an enduring length of stay. With Dutch panel data, we investigate which patterns in self-rated health can be found among immigrants shortly after their migration. We test to what extent economic, social, cultural and emotional explanations affect the changes that immigrants report in self-rated health. Based on a four-wave panel, our results support the immigrants' health decline hypothesis, since the self-rated health decreases in the first years after immigration to the Netherlands. The major change occurs between immigrants rating their health no longer as “very good,” but as “good.” Shortly after immigration, self-rated health is associated with being employed and a higher income. Hazardous work and physically heavy work decrease self-rated health. Notwithstanding these effects, social, cultural, and emotional explanations turn out to be stronger. A lack of Dutch friends, perceptions of discrimination, perceived cultural distance, and feelings of homesickness strongly affect self-rated health. Furthermore, in understanding changes in self-rated health, the effects of making contact with Dutch people and changes in the perception of discrimination are definitive. However, contact with Dutch people did not decrease and discrimination did not increase over time, making them ineligible as an explanation for overall health decrease. Only the small effect that first-borns have may count as a reason for decreased self-rated health, since many of the recent immigrants we followed started families in the first years after immigration. Our findings leave room for the coined “acculturation to an unhealthier lifestyle thesis,” and we see promise in a stronger focus on the role of unmet expectations in the first years after immigration.