(1) Background: Spouse carers of persons with dementia (PwD) are particularly vulnerable to negative outcomes of care, yet research rarely focuses on their caregiving situation. This study explores factors associated with the positive value and negative impact of caregiving in spouse carers of PwD in Sweden. (2) Methods: The study was a cross-sectional questionnaire-based survey, with a convenience sample of spouse carers of PwD (n = 163). The questionnaire addressed: care situation, carer stress, health and social well-being, relationship quality and quality of support, and contained measures of positive value and negative impact of caregiving. (3) Results: Hierarchical regression models explained 63.4% variance in positive value and 63.2% variance in negative impact of caregiving. Three variables were significant in the model of positive value: mutuality, change in emotional closeness following dementia and quality of support. Six variables were significant in the model of negative impact: years in relationship, years as carer, behavioural stress, self-rated health, emotional loneliness and change in physical intimacy following dementia. (4) Conclusions: Support to spouse carers of PwD should address the carer–care-recipient relationship quality, although different aspects of the relationship should be addressed if both the positive value of caregiving is to be enhanced and the negative impact reduced.