Benthic Archaea comprise a significant part of the total prokaryotic biomass in marine sediments. Recent genomic surveys suggest they are largely involved in anaerobic processing of organic matter but the distribution and abundance of these archaeal groups is still largely unknown. Archaeal membrane lipids composed of isoprenoid diethers or tetraethers (glycerol dibiphytanyl glycerol tetraether, GDGT) are often used as archaeal biomarkers. Here, we compare the archaeal diversity and intact polar lipid (IPL) composition in both surface (0-0.5 cm) and subsurface (10-12 cm) sediments recovered within, just below, and well below the oxygen minimum zone (OMZ) of the Arabian Sea. Archaeal 16S rRNA gene amplicon sequencing revealed a predominance of Thaumarchaeota (Marine Group I, MG-I) in oxygenated sediments. Quantification of archaeal 16S rRNA and ammonia monoxygenase (amoA) of Thaumarchaeota genes and their transcripts indicated the presence of an active in situ benthic population, which coincided with a high relative abundance of hexose phosphohexose crenarchaeol, a specific biomarker for living Thaumarchaeota. On the other hand, anoxic surface sediments within the OMZ and all subsurface sediments were dominated by archaea belonging to the Miscellaneous Crenarchaeota Group (MCG), the Thermoplasmatales and archaea of the DPANN superphylum. Members of the MCG were diverse with a dominance of subgroup MCG-12 in anoxic surface sediments. This coincided with a high relative abundance of IPL GDGT-0 with an unknown polar head group. Subsurface anoxic sediments were characterized by higher relative abundance of GDGT-0, 2 and 3 with dihexose IPL-types, as well as GDGT-0 with a cyclopentanetetraol molecule and a hexose, as well as the presence of specific MCG subgroups, suggesting that these groups could be the biological sources of these archaeal lipids.