With the aim of studying the role of contextual factors for explaining within-country variation in the vote share of the radical right-wing party, the Sweden Democrats, in the 2014 Swedish election, we specify and test hypotheses pertaining to social marginality, ethnic threat, the contact hypothesis, and the halo effect. We study the variation in the electoral share of the Sweden Democrats at two different levels simultaneously by performing multilevel analyses to account for the ways in which voting districts are clustered within municipalities. The main finding from our analyses is the support for the ethnic threat hypothesis, where the vote share of the Sweden Democrats is significantly higher in those areas that have seen an increase in the foreign-born population, and to some extent also in ethnically diverse areas, contradicting previous research on ethnic minority presence in fine-grained contexts. The expectation that the vote share for the Sweden Democrats should be higher in socioeconomic marginalized districts is partly supported, but we find no evidence of a halo effect, where ethnically homogeneous areas that are geographically close to heterogeneous districts were expected to have a higher vote share for the Sweden Democrats.