The absence of time-reversal symmetry is a fundamental property of many nonlinear time series. Here, we propose a new set of statistical tests for time series irreversibility based on standard and horizontal visibility graphs. Specifically, we statistically compare the distributions of time-directed variants of the common complex network measures degree and local clustering coefficient. Our approach does not involve surrogate data and is applicable to relatively short time series. We demonstrate its performance for paradigmatic model systems with known time-reversal properties as well m for picking up signatures of nonlinearity in neuro-physiological data.