We introduce the set of quasi-Herglotz functions and demonstrate that it has properties useful in the modelling of non-passive systems. The linear space of quasi-Herglotz functions constitutes a natural extension of the convex cone of Herglotz functions. It consists of differences of Herglotz functions and we show that several of the important properties and modelling perspectives are inherited by the new set of quasi-Herglotz functions. In particular, this applies to their integral representations, the associated integral identities or sum rules (with adequate additional assumptions), their boundary values on the real axis and the associated approximation theory. Numerical examples are included to demonstrate the modelling of a non-passive gain medium formulated as a convex optimization problem, where the generating measure is modelled by using a finite expansion of B-splines and point masses.