Context. Determination of solar magnetic fields with a spatial resolution set by the diffraction limit of a telescope is difficult because the time required to measure the Stokes vector with sufficient signal-to-noise ratio is long compared to the solar evolution timescale. This difficulty becomes greater with increasing telescope size as the photon flux per diffraction-limited resolution element remains constant but the evolution timescale decreases linearly with the diffraction-limited resolution. Aims. We aim to improve magnetic field reconstruction at the diffraction limit without averaging the observations in time or space, and without applying noise filtering. Methods. The magnetic field vector tends to evolve more slowly than the temperature, velocity, or microturbulence. We exploit this by adding temporal regularisation terms for the magnetic field to the linear least-squares fitting used in the weak-field approximation, as well as to the Levenberg-Marquardt algorithm used in inversions. The other model parameters are allowed to change in time without constraints. We infer the chromospheric magnetic field from Ca II 854.2 nm observations using the weak field approximation and the photospheric magnetic field from Fe I 617.3 nm observations, both with and without temporal regularisation. Results. Temporal regularisation reduces the noise in the reconstructed maps of the magnetic field and provides a better coherency in time in both the weak-field approximation and Milne-Eddington inversions. Conclusions. Temporal regularisation markedly improves magnetic field determination from spatially and temporally resolved observations.