This dataset contains underwater hyperspectral imagery that can be used by researchers in the domains of computer vision, machine learning, remote sensing and coral reef ecology. A diver-operated hyperspectral imaging system (HyperDiver) was used to survey 147 transects at 8 coral reef sites around the Caribbean island of Curaçao. The proximal sensing approach produced cm-scale images of more than 2.2 billion points of detailed optical spectra. Of these, more than 10 million data points have been annotated for benthic taxonomic identity with hierarchical labels. In addition to HyperDiver survey data, we also include images and annotations from traditional (color photo) quadrat surveys conducted along 23 of the 147 transects, which enables comparative reef description between two types of reef survey methods.