The Galactic Center Excess (GCE) in GeV gamma rays has been debated for over a decade, with the possibility that it might be due to dark matter annihilation or undetected point sources such as millisecond pulsars (MSPs). This study investigates how the gamma-ray emission model (γEM) used in Galactic center analyses affects the interpretation of the GCE's nature. To address this issue, we construct an ultra-fast and powerful inference pipeline based on convolutional Deep Ensemble Networks. We explore the two main competing hypotheses for the GCE using a set of γEMs with increasing parametric freedom. We calculate the fractional contribution (f_src) of a dim population of MSPs to the total luminosity of the GCE and analyze its dependence on the complexity of the γEM. For the simplest γEM, we obtain f_src = 0.10 ± 0.07, while the most complex model yields f_src = 0.79 ± 0.24. In conclusion, we find that the statement about the nature of the GCE (dark matter or not) strongly depends on the assumed γEM. The quoted results for f_src do not account for the additional uncertainty arising from the fact that the observed gamma-ray sky is out-of-distribution concerning the investigated γEM iterations. We quantify the reality gap between our γEMs using deep-learning-based One-Class Deep Support Vector Data Description networks, revealing that all employed γEMs have gaps to reality. Our study casts doubt on the validity of previous conclusions regarding the GCE and dark matter, and underscores the urgent need to account for the reality gap and consider previously overlooked "out of domain" uncertainties in future interpretations.