We present an extensive numerical study of the time irreversibility of the dynamics of heavy inertial particles in three-dimensional, statistically homogeneous, and isotropic turbulent flows. We show that the probability density function (PDF) of the increment, W(tau), of a particle's energy over a time scale tau is non-Gaussian, and skewed toward negative values. This implies that, on average, particles gain energy over a period of time that is longer than the duration over which they lose energy. We call this slow gain and fast loss. We find that the third moment of W(tau) scales as tau(3) for small values of tau. We show that the PDF of power-input p is negatively skewed too; we use this skewness Ir as a measure of the time irreversibility and we demonstrate that it increases sharply with the Stokes number St for small St; this increase slows down at St similar or equal to 1. Furthermore, we obtain the PDFs of t(+) and t(-), the times over which p has, respectively, positive or negative signs, i.e., the particle gains or loses energy. We obtain from these PDFs a direct and natural quantification of the slow gain and fast loss of the energy of the particles, because these PDFs possess exponential tails from which we infer the characteristic loss and gain times t(loss) and t(gain), respectively, and we obtain t(loss) < t(gain) for all the cases we have considered. Finally, we show that the fast loss of energy occurs with greater probability in the strain-dominated region than in the vortical one; in contrast, the slow gain in the energy of the particles is equally likely in vortical or strain-dominated regions of the flow.