Maternally inherited mitochondrial DNA (mtDNA) has been used extensively to determine genetic diversity and to guide genetic resource conservation. It is hypothesized that local populations of Chinese Yellow Cattle (Bos taurus) in the Loess Plateau of Western China were produced from Zaosheng cattle, and estimates of genetic diversity in Zaosheng and other derived populations are needed to assist in getting more detailed information about genetic resource conservation. Samples from Qinchuan cattle (QC, n = 171), Zaosheng cattle (ZS, n = 184), Pingliang native cattle (PL, n = 112), and Guyuan native cattle (GY, n = 75) were analyzed using mtDNA D-loop analytical techniques. A total of 140 variable sites and 244 haplotypes were identified. Among the QC, ZS, PL and GY populations, the diversity of haplotypes (0.946 ± 0.012, 0.976 ± 0.005, 0.966 ± 0.010, and 0.975 ± 0.009, respectively), the average number of nucleotide differences (16.312, 13.685, 14.503, and 13.778, respectively) and nucleotide diversity (0.02661, 0.02236, 0.02370, and 0.02248, respectively) were determined. There were 202 unique haplotypes found in four populations: 56 in QC, 71 in ZS, 42 in PL, and 33 in GY. Results from this research indicated that the genetic diversity of QC was lower than that of ZS, PL and GY. Results also suggested that, based on number of shared haplotypes, Qinchuan, Pingliang, and Guyuan cattle were descended from Zaosheng cattle and gradually formed three distinct maternal branches with Pingliang and Guyuan apparently maintaining the genetic diversity of Zaosheng. Identification of unique haplotypes within these populations provided a basis for further cattle genetic resource assessment of diversity and conservation of native cattle populations in the Loess Plateau region of western China.