Olfactory deficits are hypothesized to precede cognitive decline and be independently associated with future dementia. Conversely, the concurrency of cognitive and olfactory impairments is expected to represent an advanced stage, associated with shorter time to diagnosis. Limited research has examined the association of isolated and concurrent cognitive and olfactory impairments with incident dementia. We aimed to estimate the 12-year dementia hazard for cognitive impairment no dementia (CIND), olfactory dysfunction (OD), and their combination in a population-based cohort of older adults. We classified 2406 participants from the Swedish National Study on Aging and Care in Kungsholmen (SNAC-K) based on baseline CIND and OD. Dementia hazard was estimated with Cox regressions for the whole period and two timeframes (baseline to 6-year follow-up and 6- to 12-year follow-up), and time until receiving a diagnosis via Laplace regressions. CIND+OD was associated with increased hazard ratio (HR) of dementia over 6 years (HR 11.38; 95% CI 6.70, 19.32; p < 0.001), higher for amnestic CIND+OD (HR 22.23; 95% CI 11.79, 41.90; p < 0.001). Isolated CIND was associated with dementia closest to baseline (HR 3.38; 95% CI 1.75, 6.49; p < 0.001), while isolated OD was associated with dementia closest (HR 2.56; 95% CI 1.48, 4.43; p < 0.001) and furthest (HR 2.12; 95% CI 1.41, 3.19; p < 0.001) to baseline. CIND+OD received their dementia diagnosis 3 years earlier. This study demonstrated that individuals with both cognitive and olfactory impairments have a higher short-term risk of progression to dementia and that OD may be a valuable early marker of dementia on its own.