Ungulate species have experienced severe declines over the past centuries through overharvesting and habitat loss. Even if many game species have recovered thanks to strict hunting regulation, the genome-wide impacts of overharvesting are still unclear. Here, we examine the temporal and geographical differences in genome-wide diversity in moose (Alces alces) over its whole range in Sweden by sequencing 87 modern and historical genomes. We found limited impact of the 1900s near-extinction event but local variation in inbreeding and load in modern populations, as well as suggestion of a risk of future reduction in genetic diversity and gene flow. Furthermore, we found candidate genes for local adaptation, and rapid temporal allele frequency shifts involving coding genes since the 1980s, possibly due to selective harvesting. Our results highlight that genomic changes potentially impacting fitness can occur over short time scales and underline the need to track both deleterious and selectively advantageous genomic variation.