This paper reviews the challenges in the water supply provision, water source availability and quality and the distribution approaches in rural Sabah. The main challenges to provide potable water in Sabah is the variance in terrain and geographical distance between populated regions. Review reveals that other than the river water, average annual precipitation of 3000 millimetres (mm) could be harvested for domestic and agricultural purposes. Numbers of aquifer uncovered in the eastern and western region of Sabah with underlying sandstone and Quaternary Alluvium have significant potential for groundwater reservoirs. Aquifer along the coastal areas and islands around Sabah also gives sufficient potable water supplies. Minimal pollutant content was found in all water sources and acceptable under the National Water Standard of Malaysia, except for contaminants coming from septic tanks and agricultural activities. A decentralized water system is more beneficial for Sabah’s rural areas. Smaller scaled plants are flexible to collect from any water sources and treat at the point of use. Expenditure is significantly decreased by a shorter distribution network and lower installation and maintenance cost. Nonetheless, the treatment utilized may be limited to a simpler process as semiskilled or un-skilled personnel will be required to operate and maintain the system.