Despite acute interest in the dynamics of non-Hermitian systems, there is a lack of consensus in the mathematical formulation of non-Hermitian quantum mechanics in the community. Different methodologies are used in the literature to study non-Hermitian dynamics. This ranges from consistent frameworks like biorthogonal quantum mechanics and metric approach characterized by modified inner products, to normalization by time-dependent norms inspired by open quantum systems. In this work, we systematically explore the similarities and differences among these various methods. Utilizing illustrative models with exact solutions, we demonstrate that these methods produce not only quantitatively different results but also distinct physical interpretations. For dissipative systems where non-Hermiticity arises as an approximation, we find that simply dividing by the norm in the 𝒫𝒯-broken regime closely aligns with the full master equation solutions. In contrast, for quantum systems where non-Hermiticity can be engineered exactly, incorporating metric dynamics is crucial for the probabilistic interpretation of quantum mechanics, necessitating the generalizations of similarity transformations and unitarity to non-Hermitian systems. This study lays the groundwork for further exploration of non-Hermitian Hamiltonians, potentially leveraging generalized transformations for novel physical phenomena.