We investigate a 2,000-year genetic transect through Scandinavia spanning the Iron Age to the present, based on 48 new and 249 published ancient genomes and genotypes from 16,638 modern individuals. We find regional variation in the timing and magnitude of gene flow from three sources: the eastern Baltic, the British-Irish Isles, and southern Europe. British-Irish ancestry was widespread in Scandinavia from the Viking period, whereas eastern Baltic ancestry is more localized to Gotland and central Sweden. In some regions, a drop in current levels of external ancestry suggests that ancient immigrants contributed proportionately less to the modern Scandinavian gene pool than indicated by the ancestry of genomes from the Viking and Medieval periods. Finally, we show that a north-south genetic cline that characterizes modern Scandinavians is mainly due to the differential levels of Uralic ancestry and that this cline existed in the Viking Age and possibly earlier.