There is growing awareness of the potential value of agricultural land for climate change mitigation. In Sweden, cropland areas have decreased by approximately 30% over recent decades, creating opportunities for these former croplands to be managed for climate change mitigation by increasing soil organic carbon (SOC) stocks. One potential land-use change is conversion of cropland to grazed grasslands, but the long-term effect of such change in management is not well understood and likely varies with soil type and site-specific conditions. Through sampling of mineral and peatland soils within a 75-year chronosequence of land converted from crop production to grazed grassland, we assessed how time since conversion, catenary position, and soil depth affected SOC storage. The SOC stocks calculated at an equivalent soil or ash mass increased through time since conversion in mineral soils at all topographic positions, at a rate of ~0.65% year−1. Soils at low topographic positions gained the most carbon. Peat SOC stock gains after conversion were large, but only marginally significant and only when calculated at an equivalent ash mass. We conclude that the conversion of mineral soil to grazed grassland promotes SOC accumulation at our sites, but climate change mitigation potential would need to be evaluated through a full greenhouse gas balance.