ATLAS measurements of two-particle correlations are presented for root s = 5.02 and 13 TeV pp collisions and for root(NN)-N-s = 5.02 TeV p + Pb collisions at the LHC. The correlation functions are measured as a function of relative azimuthal angle Delta phi, and pseudorapidity separation Delta eta, using charged particles detected within the pseudorapidity interval |eta| < 2.5. Azimuthal modulation in the long-range component of the correlation function, with | Delta eta| > 2, is studied using a template fitting procedure to remove a back-to-back contribution to the correlation function that primarily arises from hard-scattering processes. In addition to the elliptic, cos(2 Delta phi), modulation observed in a previous measurement, the pp correlation functions exhibit significant cos(3 Delta phi) and cos(4 Lambda phi) modulation. The Fourier coefficients v(n),(n) associated with the cos (n Lambda phi) modulation of the correlation functions for n = 2-4 are measured as a function of charged-particle multiplicity and charged-particle transverse momentum. The Fourier coefficients are observed to be compatible with cos(n phi) modulation of per-event singleparticle azimuthal angle distributions. The single-particle Fourier coefficients vn are measured as a function of charged-particle multiplicity, and charged-particle transverse momentum for n = 2-4. The integrated luminosities used in this analysis are, 64 nb(-1) for the root s = 13 TeV pp data, 170 nb(-1) for the root s = 5.02 TeV pp data, and 28 nb(-1) for the root(NN)-N-s = 5.02 TeV p + Pb data.