The binary neutron star merger GW170817 was the first multi-messenger event observed in both gravitational and electromagnetic waves(1,2). The electromagnetic signal began approximately two seconds post-merger with a weak, short burst of gamma rays(3), which was followed over the next hours and days by the ultraviolet, optical and near-infrared emission from a radioactively powered kilonova(4-11). Later, non-thermal rising X-ray and radio emission was observed(12,13). The low luminosity of the gamma rays and the rising non-thermal flux from the source at late times could indicate that we are outside the opening angle of the beamed relativistic jet. Alternatively, the emission could be arising from a cocoon of material formed from the interaction between a jet and the merger ejecta(13-15). Here we present late-time optical detections and deep near-infrared limits on the emission from GW170817 at 110 days post-merger. Our new observations are at odds with expectations of late-time emission from kilonova models, being too bright and blue(16,17). Instead, the emission arises from the interaction between the relativistic ejecta of GW170817 and the interstellar medium. We show that this emission matches the expectations of a Gaussian-structured relativistic jet, which would have launched a high-luminosity, short gamma-ray burst to an aligned observer. However, other jet structure or cocoon models can also match current data-the future evolution of the afterglow will directly distinguish the origin of the emission.