Species frequently differ in the number and structure of chromosomes they harbor, but individuals that are heterozygous for chromosomal rearrangements may suffer from reduced fitness. Chromosomal rearrangements like fissions and fusions can hence serve as a mechanism for speciation between incipient lineages, but their evolution poses a paradox. How can rearrangements get fixed between populations if heterozygotes have reduced fitness? One solution is that this process predominantly occurs in small and isolated populations, where genetic drift can override natural selection. However, fixation is also more likely if a novel rearrangement is favored by a transmission bias, such as meiotic drive. Here, we investigate chromosomal transmission distortion in hybrids between two wood white (Leptidea sinapis) butterfly populations with extensive karyotype differences. Using data from two different crossing experiments, we uncover that there is a transmission bias favoring the ancestral chromosomal state for derived fusions, a result that shows that chromosome fusions actually can fix in populations despite being counteracted by meiotic drive. This means that meiotic drive not only can promote runaway chromosome number evolution and speciation, but also that it can be a conservative force acting against karyotypic change and the evolution of reproductive isolation. Based on our results, we suggest a mechanistic model for why chromosome fusion mutations may be opposed by meiotic drive and discuss factors contributing to karyotype evolution in Lepidoptera.