Strong gravitational lensing of variable sources, such as quasars or supernovae, can be used to constrain cosmological parameters through a technique known as 'time-delay cosmography'. Competitive constraints on the Hubble constant have been achieved with electromagnetic (EM) observations of lensed quasars and lensed supernovae. Gravitational wave (GW) astronomy may open up a new channel for time-delay cosmography with GW signal replacing the EM one. We highlight the similarities of using GW signals to be applied to time-delay cosmography compared with EM signal. We then discuss key differences between GW and EM signals and their resulting advantages and inconveniences from the angle of the current state-of-the-art using quasars and lensed supernovae for time-delay cosmography. We identify the astrometric precision requirement of the images as a key challenge to overcome and highlight the potentially significant impact that near-perfect time-delay measurements of lensed GWs can bring to the table. This article is part of the Theo Murphy meeting issue 'Multi-messenger gravitational lensing (Part 2)'.