Theory predicts that the ability of selection and recombination to purge mutation load is enhanced if selection against deleterious genetic variants operates more strongly in males than females. However, direct empirical support for this tenet is limited, in part because traditional quantitative genetic approaches allow dominance and intermediate-frequency polymorphisms to obscure the effects of the many rare and partially recessive deleterious alleles that make up the main part of a population's mutation load. Here, we exposed the partially recessive genetic load of a population of Callosobruchus maculatus seed beetles via successive generations of inbreeding, and quantified its effects by measuring heterosis—the increase in fitness experienced when masking the effects of deleterious alleles by heterozygosity—in a fully factorial sex-specific diallel cross among 16 inbred strains. Competitive lifetime reproductive success (i.e., fitness) was measured in male and female outcrossed F₁s as well as inbred parental “selfs,” and we estimated the 4 × 4 male-female inbred-outbred genetic covariance matrix for fitness using Bayesian Markov chain Monte Carlo simulations of a custom-made general linear mixed effects model. We found that heterosis estimated independently in males and females was highly genetically correlated among strains, and that heterosis was strongly negatively genetically correlated to outbred male, but not female, fitness. This suggests that genetic variation for fitness in males, but not in females, reflects the amount of (partially) recessive deleterious alleles segregating at mutation-selection balance in this population. The population's mutation load therefore has greater potential to be purged via selection in males. These findings contribute to our understanding of the prevalence of sexual reproduction in nature and the maintenance of genetic variation in fitness-related traits.