The factorial structure of the Inventory of Callous-Unemotional traits (ICU) is still under debate in the current literature and the published models are predominantly based on the empirical results of the statistical analyses rather than on a strong theoretical background. Aimed at overcoming these limits, the current study examined a factorial structure initiated by a theoretical framework for the parent-version of the ICU, based on a community sample of Belgian children aged 3–9 ( N = 437; M age = 5.59; 54.7% boys). Further, the current study investigated measurement invariance across age and gender, and the external validity of this structure. Confirmatory factor analysis (CFA) indicated that empirical factor models from the current literature demonstrated a relatively poor fit to the data. Alternative models were built based on theory, specifically criteria from the DSM-V specifier “with limited prosocial emotions.” CFA supported an 18-item second order model with three first order factors (Lack of conscience, Unconcern about performance, Lack of emotional expression), a second order latent factor (General dimension of CU traits) and a methodological factor encompassing negatively worded items. Results supported measurement invariance across child gender, and to a lesser extent across age. As expected, the general dimension correlated with measures of aggressive behavior, attention problems, internalizing behavior and empathy. The Lack of emotional expression subfactor showed a different pattern of associations in comparison to the two other subfactors. The implications of these findings are discussed, specifically in relation to the DSM-V LPE specifier.