We describe an improved in-situ calibration of the single-photoelectron charge distributions for each of the in-ice Hamamatsu Photonics R7081-02[MOD] photomultiplier tubes in the IceCube Neutrino Observatory. The characterization of the individual PMT charge distributions is important for PMT calibration, data and Monte Carlo simulation agreement, and understanding the effect of hardware differences within the detector. We discuss the single photoelectron identification procedure and how we extract the single-photoelectron charge distribution using a deconvolution of the multiple-photoelectron charge distribution.