Collisionless, magnetized turbulence offers a promising framework for the generation of nonthermal high-energy particles in various astrophysical sites. Yet, the detailed mechanism that governs particle acceleration has remained subject to debate. By means of 2D and 3D particle-in-cell, as well as 3D (incompressible) magnetohydrodynamic (MHD) simulations, we test here a recent model of nonresonant particle acceleration in strongly magnetized turbulence [Lemoine, Phys. Rev. D 104, 063020 (2021)], which ascribes the energization of particles to their continuous interaction with the random velocity flow of the turbulence, in the spirit of the original Fermi model. To do so, we compare, for a large number of particles that were tracked in the simulations, the predicted and the observed histories of particles momenta. The predicted history is that derived from the model, after extracting from the simulations, at each point along the particle trajectory, the three force terms that control acceleration: the acceleration of the field line velocity projected along the field line direction, its shear projected along the same direction, and its transverse compressive part. Overall, we find a clear correlation between the model predictions and the numerical experiments, indicating that this nonresonant model can successfully account for the bulk of particle energization through Fermi-type processes in strongly magnetized turbulence. We also observe that the parallel shear contribution tends to dominate the physics of energization in the particle-in-cell simulations, while in the magnetohydrodynamic incompressible simulation, both the parallel shear and the transverse compressive term provide about equal contributions.