Policy reform evaluations are an important source of knowledge regarding the (in)effectiveness of policies and have become increasingly influential in informing policymaking. A key advantage of these studies lies in their careful description of a single reform, observing outcomes before and after the reform was implemented, and making precise statements about the effectiveness of the policy reform among its beneficiaries. However, the very nature of reform studies also entails a number of limitations that are important to reflect on – particularly given their popularity in the context of evidence-based policymaking. We highlight five blind spots relevant to evidence-based policymaking, by critically reflecting on a focused literature review of reform studies of early childhood education and care (ECEC) policies published between 2000 and 2021, conducted by the authors and commissioned by the European Commission. The critical reflection highlights: (a) the context-specific nature of reform studies; (b) the limited focus on reducing inequality in the use and benefits of ECEC; (c) a focus on short-term outcomes; (d) a focus on individual-level rather than macro-level outcomes; and (e) various forms of publication bias.