Mating system transitions are common in flowering plants and have profound effects on genetic diversity, adaptation, and speciation. This thesis investigates the evolution of the S-locus supergene governing distyly, a floral polymorphism promoting outcrossing, and examines how mating system shifts and demography shape genome evolution. In Chapter I, we generated genome assemblies and annotations for two wild flax species, the distylous Linum perenne and style length polymorphic Linum grandiflorum. Based on these assemblies, we characterized S-locus architecture and compared it to that of the previously described S-locus in the distylous Linum tenue. Analyses revealed that the dominant S haplotype was longer than the recessive s haplotype, resulting in hemizygosity in the short-styled floral morph. Comparing the S-locus of all three species, we found extensive gene turnover, yet conservation of two candidate genes for distyly, TSS1 and WDR-44, which are likely to regulate style and anther filament length and thus morph differentiation. Molecular dating placed the likely origin of the supergene near the root of Linum (~33 Mya). In Chapter II, we investigated distyly breakdown using comparative genomic, population genetic and transcriptomic analyses of L. tenue and its homostylous, self-compatible relative L. trigynum. The S-locus region is homozygous in L. trigynum, and downregulation of WDR-44 likely enables autonomous selfing. Expression patterns and crossing experiments confirmed WDR-44 as a candidate for male self-incompatibility control, providing a molecular explanation for distyly breakdown. In Chapter III, we conducted a comparative genomics and phylogenomic study across repeated shifts from distyly to homostyly and tested whether repeated transitions to self-compatibility showed consistent genomic signatures. Nuclear and chloroplast data revealed relaxed purifying selection and elevated nonsynonymous substitution rates in homostylous species, although the magnitude differed between clades. An investigation in the homostylous self-compatible Linum leonii revealed that loss of distyly was not associated with a shift to high self-fertilization. Demographic history inference nevertheless suggested that L. leonii has undergone population size reductions, reinforcing the genomic effects of self-compatibility. In Chapter IV, we assessed how mating system and demographic history interact to shape patterns of genomic variation and selection efficacy within Arabis alpina, a crucifer species exhibiting intraspecific mating system variation from outcrossing to predominant selfing. By combining population genomic analyses and demographic history modelling, we showed that while increased selfing rates can explain most of the observed patterns, in Scandinavia reduced nucleotide diversity is a combined result of the mating system transition and demographic history. Across study systems, self-compatible species or populations exhibited reduced nucleotide diversity, effective population sizes, and efficacy of selection, supporting theoretical predictions of reduced evolutionary potential under selfing. Collectively, my findings demonstrate that a hemizygous S-locus supergene underlies distyly in Linum, that genetic changes affecting expression of S-locus genes are associated with loss of distyly, and that transitions to selfing leave genomic signatures of reduced diversity and selection efficacy across evolutionary scales.