Constipation, a disorder of bowel movements, is among the most frequent gastrointestinal complaints in Western countries. Dietary constituents such as inadequate fiber intake have been related to constipation, but discrepancies exist in the findings regarding dietary factors. This study investigated the association between dietary patterns and bowel movements in adults living in Luxembourg. Data from 1431 participants from ORISCAV-LUX 2 (a cross-sectional survey) who completed a 174-item food frequency questionnaire (FFQ) were analyzed. A questionnaire-based constipation score was assessed by a validated scoring system. Confounders such as physical activity and serum/urine indicators were assessed. Women had higher constipation scores than men (p < 0.001). In food group-based regression models, a negative association was found between higher constipation score and intake of grains (Beta = −0.62, 95%CI: −1.18, −0.05) and lipid-rich foods (Beta = −0.84, 95%CI: −1.55, −0.13), while a positive association was found for sugary products (Beta = 0.54, 95%CI: 0.11, 0.97) (p < 0.05). In a nutrient-based regression model, a positive association was found between constipation score and total energy (Beta = 5.24, 95%CI: 0.37, 10.11) as well as sodium intake (Beta = 2.04, 95%CI: 0.21, 3.87), and a negative one was found for total fats (Beta = −4.17, 95%CI: −7.46, −0.89) and starch (Beta = −2.91, 95%CI: −4.47, −1.36) (p < 0.05). Interestingly, neither fruits and vegetables or dietary fiber were significantly associated with constipation. Thus, grains, lipid-rich foods, total fats and starch were associated with a lower constipation score, while sugary products, sodium, and higher energy intake were correlated with higher constipation.