Guided by previous non-perturbative lattice simulations of a two-step electroweak phase transition, we reformulate the perturbative analysis of equilibrium thermodynamics for generic cosmological phase transitions in terms of effective field theory (EFT) expansions. Based on thermal scale hierarchies, we argue that the scale of many interesting phase transitions is in-between the soft and ultrasoft energy scales, which have been the focus of studies utilising high-temperature dimensional reduction. The corresponding EFT expansions provide a handle to control the perturbative expansion, and allow us to avoid spurious infrared divergences, imaginary parts, gauge dependence and renormalisation scale dependence that have plagued previous studies. As a direct application, we present a novel approach to two-step electroweak phase transitions, by constructing separate effective descriptions for two consecutive transitions. Our approach provides simple expressions for effective potentials separately in different phases, a numerically inexpensive method to determine thermodynamics, and significantly improves agreement with the non-perturbative lattice simulations.