Thermal dark matter models generically include the prompt creation of highly concentrated dark matter cusps in the early Universe. Recent studies find that these cusps can survive to the present day, as long as they do not fall into extremely dense regions of baryonic structure. In this work, we build models of dark matter annihilation within the prompt cusps that reside in galaxy clusters, showing that they dominate the total 𝛾-ray annihilation signal. Using 15 years of Fermi Large Area Telescope data, we find no evidence for a 𝛾-ray excess from these sources, and set strong constraints on annihilating dark matter. These constraints generically rule out the thermal annihilation cross section to the 𝑏⁢¯𝑏 channel for dark matter masses below  ∼200  GeV.