Background: Epstein-Barr virus (EBV) and cytomegalovirus (CMV) are ubiquitous and persistent herpesviruses commonly acquired during childhood. Both viruses have a significant impact on the immune system, especially through mediating the establishment of cellular immunity, which keeps these viruses under control for life. Far less is known about how these viruses influence B-cell responses. Objectives: To evaluate the impact of latent EBV and CMV infection on rubella- and measles-specific antibody responses as well as on the B-cell compartment in a prospective birth cohort followed during the first 10 years of life. Methods: IgG titers against rubella and measles vaccines were measured in plasma obtained from the same donors at 2, 5, and 10 years of age. Peripheral B-cell subsets were evaluated ex vivo at 2 and 5 years of age. Factors related to optimal B-cell responses including IL-21 and CXCL13 levels in plasma were measured at all-time points. Results: EBV carriage in the absence of CMV associated with an accelerated decline of rubella and measles-specific IgG levels (p = 0.003 and p = 0.019, respectively, linear mixed model analysis), while CMV carriage in the absence of EBV associated with delayed IgG decay over time for rubella (p = 0.034). At 5 years of age, EBV but not CMV latency associated with a lower percentage of plasmablasts, but higher IL-21 levels in the circulation. Conclusion: Our findings suggest that EBV carriage in the absence of CMV influences the B-cell compartment and the dynamics of antibody responses over time during steady state in the otherwise healthy host.