Africa’s black (Diceros bicornis) and white (Ceratotherium simum) rhinoceros are closely related sister-taxa that evolved highly divergent obligate browsing and grazing feeding strategies. Although their precursor species Diceros praecox and Ceratotherium mauritanicum appear in the fossil record ∼5.2 Ma, by 4 Ma both were still mixed feeders, and were even spatiotemporally sympatric at several Pliocene sites in what is today Africa’s Rift Valley. Here, we ask whether or not D. praecox and C. mauritanicum were reproductively isolated when they came into Pliocene secondary contact. We sequenced and de novo assembled the first annotated black rhinoceros reference genome and compared it with available genomes of other black and white rhinoceros. We show that ancestral gene flow between D. praecox and C. mauritanicum ceased sometime between 3.3 and 4.1 Ma, despite conventional methods for the detection of gene flow from whole genome data returning false positive signatures of recent interspecific migration due to incomplete lineage sorting. We propose that ongoing Pliocene genetic exchange, for up to 2 My after initial divergence, could have potentially hindered the development of obligate feeding strategies until both species were fully reproductively isolated, but that the more severe and shifting paleoclimate of the early Pleistocene was likely the ultimate driver of ecological specialization in African rhinoceros.