Ultraviolet absorption line spectroscopy is a sensitive diagnostic for the properties of interstellar and circumgalactic gas. Down-the-barrel observations, where the absorption is measured against the galaxy itself, are commonly used to study feedback from galactic outflows and to make predictions about the leakage of H i ionizing photons into the intergalactic medium. Nonetheless, the interpretation of these observations is challenging, and observational compromises are often made in terms of signal-to-noise ratio, spectral resolution, or the use of stacking analyses. In this paper, we present a novel quantitative assessment of UV absorption line measurement techniques by using mock observations of a hydrodynamical simulation. We use a simulated galaxy to create 22,500 spectra in the commonly used Si ii lines while also modeling the signal-to-noise ratio and spectral resolution of recent rest-frame UV galaxy surveys at both high and low redshifts. We show that the residual flux of absorption features is easily overestimated for single line measurements and for stacked spectra. Additionally, we explore the robustness of the partial covering model for estimating column densities from spectra and find underpredictions on an average of 1.25 dex. We show that the underprediction is likely caused by high-column-density sight lines that are optically thick to dust making them invisible in UV spectra.