The baroclinic annular mode (BAM) is a leading-order mode of the eddy kinetic energy in the Southern Hemisphere exhibiting oscillatory behaviour at intraseasonal time-scales. The oscillation mechanism has been linked to transient eddy–mean flow interactions which remain poorly understood. Here we demonstrate that the finite memory effect in eddy-heat flux dependence on the large-scale flow can explain the origin of the BAM's oscillatory behaviour. We represent the eddy memory effect by a delayed integral kernel that leads to a generalized Langevin equation for the planetary-scale heat equation. Using a mathematical framework for the interactions between planetary- and synoptic-scale motions, we derive a reduced dynamical model of the BAM – a stochastically forced oscillator with a period proportional to the geometric mean between the eddy memory time-scale and the diffusive eddy equilibration time-scale. Our model provides a formal justification for the previously proposed phenomenological model of the BAM and could be used to explicitly diagnose the memory kernel and improve our understanding of transient eddy–mean flow interactions in the atmosphere.