The longitudinal effects of perceived stress on measures of memory and two other cognitive functions (word fluency, visuospatial ability) in a middle-aged sample (40–60 years, M age = 47.1 years, SD = 6.1 years; n = 192) were examined. A group describing themselves as stressed in general at baseline, and at follow-up measurement 5 and 10 years later (n = 96) was compared with a matched (age, sex) low-stress group (n = 96). The results revealed more depressive symptoms over time in the high-stress group. With regard to memory, a dissociation between subjective and objective measures was observed. Specifically, participants in the high-stress group rated their memory as worse over time as compared with controls, and reported a higher frequency of occurrence of everyday memory failures, effects partly independent of depressive symptoms. However, the groups did not differ in terms of objective episodic memory performance, word fluency or block design performance, with stable levels of performance over time regardless of perceived stress. The lack of effects of stress on cognitive performance is discussed in the light of factors such as stress level, age of the participants, and other individual difference factors.