Whenever the integrand of a gauge-theory loop amplitude can be arranged into a form where the Bern-Carrasco-Johansson duality between color and kinematics is manifest, a corresponding gravity integrand can be obtained simply via the double-copy procedure. However, finding such gauge-theory representations can be challenging, especially at high loop orders. Here, we show that we can, instead, start from generic gauge-theory integrands, where the duality is not manifest, and apply a modified double-copy procedure to obtain gravity integrands that include contact terms generated by violations of dual Jacobi identities. We illustrate this with three-, four- and five-loop examples in N = 8 supergravity.