Quantification of the soil organic carbon (SOC) content over large areas is mandatory to obtain accurate soil characterization and classification, which can improve site-specific management at local or regional scales. In this context, soil spectroscopy is a well-consolidated and widespread method to estimate soil variables, and in particular SOC content, at a low cost for routine analysis. The increasing number of large soil spectral libraries collected worldwide reflects the importance of spectroscopy in soil science. These large libraries contain soil samples derived from a large number of pedological regions and thus from different parent materials and soil types. In the light of the huge variation in the spectral responses to SOC content and composition, a rigorous process is necessary to subdivide large spectral libraries to avoid calibration with global models that fail to predict local variation in SOC content. Here, we propose to classify the European LUCAS topsoil database with a cluster analysis based on a large number of soil properties. The soil samples collected from arable land in the LUCAS database were chosen to apply a standardized multivariate calibration approach, valid for large areas, to calibrate local models without the need for further field and laboratory work. Cluster analysis detected seven soil classes and the samples belonging to each class were used to calibrate specific partial least squares regression (PLSR) models to estimate SOC content in three spectral libraries collected in Belgium and Luxembourg. Soil organic carbon was predicted with good accuracy, both within each library (root mean square error (RMSE), 1.2–5.1 g kg−1; ratio of performance to prediction (RPD), 1.41–2.24) and for the samples of the three libraries together (RMSE, 3.7 g kg−1; RPD, 2.54). The proposed approach could enable SOC to be estimated for arable soils in Europe with only the spectra of soil samples and without the need for laboratory analyses.