The paleoclimate record provides a test-bed in which climate models can be evaluated under conditions of substantial CO₂ change; however, these data are typically under-used in the process of model development and evaluation. Here, we use a set of metrics based on paleoclimate proxy observations to evaluate climate models under three past time periods. We find that the latest CMIP6/PMIP4 ensemble mean does a remarkably good job of simulating the global mean surface air temperatures of these past periods, and is improved on CMIP5/PMIP3, implying that the modern climate sensitivity of the CMIP6/PMIP4 model ensemble mean is consistent with the paleoclimate record. However, some models, in particular those with very high or very low climate sensitivity, simulate paleo temperatures that are outside the uncertainty range of the paleo proxy temperature data; in this regard, the paleo data can provide a more stringent constraint than data from the historical record. There is also consistency between models and data in terms of polar amplification, with amplification increasing with increasing global mean temperature across all three time periods. The work highlights the benefits of using the paleoclimate record in the model development and evaluation cycle, in particular for screening models with too-high or too-low climate sensitivity across a range of CO₂ concentrations.