This paper describes sexual harassment risks from two groups of perpetrators in Swedish nationally representative survey data. We find that both women and men face elevated risks of intraorganizational sexual harassment from colleagues or managers when working in occupations dominated by the opposite sex. In contrast, they face elevated risks of extraorganizational harassment from outside groups like patients, customers, or clients when working in women-dominated occupations. Amounts of interpersonal contacts with potential perpetrators appears to be a relevant mechanism behind these patterns: gender minorities in occupations have more workplace interactions with individuals of the opposite sex, and workers in women-dominated occupations have more interactions with outside groups. The results also suggest that gender norms on occupation choice further contribute to the victimization of gender minorities at the hands of both perpetrator groups.