The origins of upland traditional rice varieties in Sabah, Malaysian Borneo and their domestications are still being debated until today, despite the fact that rice planting has been practiced for centuries in this region. We report the genetic diversity of upland traditional rice varieties in Sabah targeting the mitochondrial cytochrome c oxidase 3 (COX3) gene in this study. The upland traditional rice germplasms were collected from three divisions in Sabah, namely the Interior Division (ID), Sandakan Division (SD), and West Coast Division (WCD). Polymerase chain reaction was conducted to amplify the COX3 gene of these germplasms, cloned into a vector, and subjected to direct sequencing. The genetic diversity of the aligned COX3 coding sequences among rice varieties were analyzed. This study identified an amino acid variant (S186L) in the COX3 gene that enables the differentiation between Japonica and Indica rice groups, and almost all the upland traditional rice varieties in this study shared the same ancestor as the Japonica group. The genetic diversity among the upland traditional rice varieties was relatively low and the genetic variations in the COX3 gene had undergone positive selection. A total of 14 monomorphic amino acid variations were identified. Rice varieties from ID were more genetically similar to those from WCD but genetically diverse to rice varieties from SD. In conclusion, the S186L amino acid variant in the COX3 gene is a reliable molecular marker to differentiate between Japonica and Indica rice groups in this study. The genetic knowledge of rice varieties in this study could be utilized to broaden the genetic diversity of rice in a local and international scale.