The dynamics of small, yet heavy, identical particles in turbulence exhibits singularities, called caustics, that lead to large fluctuations in the spatial particle-number density, and in collision velocities. For large particle inertia, the fluid velocity at the particle position is essentially a white-noise signal and caustic formation is analogous to Kramers escape. Here we show that caustic formation at small particle inertia is different. Caustics tend to form in the vicinity of particle trajectories that experience a specific history of fluid-velocity gradients, characterized by low vorticity and a violent strain exceeding a large threshold. We develop a theory that explains our findings in terms of an optimal path to caustic formation that is approached in the small inertia limit.