Handling Small Calibration Sets in Mondrian Inductive Conformal Regressors
2015 (English)In: Statistical Learning and Data Sciences: Third International Symposium, SLDS 2015 Egham, UK, April 20–23, 2015 Proceedings / [ed] Alexander Gammerman, Vladimir Vovk, Harris Papadopoulos, Cham: Springer, 2015, 271-280 p.Conference paper (Refereed)Text
In inductive conformal prediction, calibration sets must contain an adequate number of instances to support the chosen confidence level. This problem is particularly prevalent when using Mondrian inductive conformal prediction, where the input space is partitioned into independently valid prediction regions. In this study, Mondrian conformal regressors, in the form of regression trees, are used to investigate two problematic aspects of small calibration sets. If there are too few calibration instances to support the significance level, we suggest using either extrapolation or altering the model. In situations where the desired significance level is between two calibration instances, the standard procedure is to choose the more nonconforming one, thus guaranteeing validity, but producing conservative conformal predictors. The suggested solution is to use interpolation between calibration instances. All proposed techniques are empirically evaluated and compared to the standard approach on 30 benchmark data set . The results show that while extrapolation often results in invalid models, interpolation works extremely well and provides increased efficiency with preserved empirical validity.
Place, publisher, year, edition, pages
Cham: Springer, 2015. 271-280 p.
, Lecture Notes in Artificial Intelligence, ISSN 0302-9743
Research subject Computer and Systems Sciences
IdentifiersURN: urn:nbn:se:su:diva-124665ISBN: 978-3-319-17090-9OAI: oai:DiVA.org:su-124665DiVA: diva2:890500
Third International Symposium, SLDS 2015, Egham, UK, April 20–23, 2015.