We study laser-assisted photodetachment time delays by attosecond pulse trains from the closed-shell negative ions F- and Cl. We investigate the separability of the delay into two contributions: (i) the Wigner-like delay associated with one-photon ionization by the attosecond pulse train and (ii) the delay associated with the exchange of an additional laser photon in the presence of the potential of the remaining target. Based on the asymptotic form of the wave packet, the latter term is expected to be negligible because the ion is neutralized, leading to a vanishing laser-ion interaction with increasing electron-atom separation. While this asymptotic behavior is verified at high photoelectron energies, we also quantify sharp deviations at low photoelectron energies. Further, these low-energy delays are clearly different for the two studied anions, indicating a breakdown of the universality of laser-ion-induced delays. The fact that the short-range potential can induce a delay of as much as 50 as can have implications for the interpretation of delay measurements also in other systems that lack long-range potential.