Sheep are among the earliest domesticated livestock species, with a wide variety of breeds present today. However, it remains unclear how far back this diversity goes, with formal documentation only dating back a few centuries. North European short-tailed (NEST) breeds are often assumed to be among the oldest domestic sheep populations, even thought to represent relicts of the earliest sheep expansions during the Neolithic period reaching Scandinavia <6,000 years ago. This study sequenced the genomes (up to 11.6X) of five sheep remains from the Baltic islands of Gotland and Åland, dating from the Late Neolithic (∼4,100 cal BP) to historical times (∼1,600 CE). Our findings indicate that these ancient sheep largely possessed the genetic characteristics of modern NEST breeds, suggesting a substantial degree of long-term continuity of this sheep type in the Baltic Sea region. Despite the wide temporal spread, population genetic analyses show high levels of affinity between the ancient genomes and they also exhibit relatively high genetic diversity when compared to modern NEST breeds, implying a loss of diversity in most breeds during the last centuries associated with breed formation and recent bottlenecks. Our results shed light on the development of breeds in Northern Europe specifically as well as the development of genetic diversity in sheep breeds, and their expansion from the domestication center in general.