Normal brain aging is a multidimensional process that includes deterioration in various brain structures and functions, with large heterogeneity in patterns and rates of decline. Sex differences have been reported for various cognitive and brain parameters, but little is known in relation to neuromodulatory aspects of brain aging. We examined sex differences in dopamine D2-receptor (D2DR) availability in relation to episodic memory, but also, grey-matter volumes, white-matter lesions, and cerebral perfusion in healthy older adults (n = 181, age: 64-68 years) from the Cognition, Brain, and Aging study. Women had higher D2DR availability in midbrain and left caudate and putamen, as well as superior episodic memory performance. Controlling for left caudate D2DR availability attenuated sex differences in memory performance. In men, lower left caudate D2DR levels were associated with lower cortical perfusion and higher burden of white-matter lesions, as well as with episodic memory performance. However, sex was not a significant moderator of the reported links to D2DR levels. Our findings suggest that sex differences in multiple associations among DA receptor availability, vascular factors, and structural connectivity contribute to sex differences in episodic memory. Future longitudinal studies need to corroborate these patterns by lead-lag associations.