Effective Utilization of Data in Inductive Conformal Prediction using Ensembles of Neural Networks
2013 (English)In: The 2013 International Joint Conference on Neural Networks (IJCNN): Proceedings, IEEE conference proceedings, 2013, 1-8 p.Conference paper (Refereed)
Conformal prediction is a new framework producing region predictions with a guaranteed error rate. Inductive conformal prediction (ICP) was designed to significantly reduce the computational cost associated with the original transductive online approach. The drawback of inductive conformal prediction is that it is not possible to use all data for training, since it sets aside some data as a separate calibration set. Recently, cross-conformal prediction (CCP) and bootstrap conformal prediction (BCP) were proposed to overcome that drawback of inductive conformal prediction. Unfortunately, CCP and BCP both need to build several models for the calibration, making them less attractive. In this study, focusing on bagged neural network ensembles as conformal predictors, ICP, CCP and BCP are compared to the very straightforward and cost-effective method of using the out-of-bag estimates for the necessary calibration. Experiments on 34 publicly available data sets conclusively show that the use of out-of-bag estimates produced the most efficient conformal predictors, making it the obvious preferred choice for ensembles in the conformal prediction framework.
Place, publisher, year, edition, pages
IEEE conference proceedings, 2013. 1-8 p.
Research subject Computer and Systems Sciences
IdentifiersURN: urn:nbn:se:su:diva-100717DOI: 10.1109/IJCNN.2013.6706817ISBN: 978-1-4673-6128-6OAI: oai:DiVA.org:su-100717DiVA: diva2:695727
The 2013 International Joint Conference on Neural Networks (IJCNN), Dallas, Texas, USA, 4-9 August 2013