Change search
ReferencesLink to record
Permanent link

Direct link
Obtaining accurate and comprehensible classifiers using oracle coaching
Stockholm University, Faculty of Social Sciences, Department of Computer and Systems Sciences.
2012 (English)In: Intelligent Data Analysis, ISSN 1088-467X, E-ISSN 1571-4128, Vol. 16, no 2, 247-263 p.Article in journal (Refereed) Published
Abstract [en]

While ensemble classifiers often reach high levels of predictive performance, the resulting models are opaque and hence do not allow direct interpretation. When employing methods that do generate transparent models, predictive performance typically has to be sacrificed. This paper presents a method of improving predictive performance of transparent models in the very common situation where instances to be classified, i.e., the production data, are known at the time of model building. This approach, named oracle coaching, employs a strong classifier, called an oracle, to guide the generation of a weaker, but transparent model. This is accomplished by using the oracle to predict class labels for the production data, and then applying the weaker method on this data, possibly in conjunction with the original training set. Evaluation on 30 data sets from the UCI repository shows that oracle coaching significantly improves predictive performance, measured by both accuracy and area under ROC curve, compared to using training data only. This result is shown to be robust for a variety of methods for generating the oracles and transparent models. More specifically, random forests and bagged radial basis function networks are used as oracles, while J48 and JRip are used for generating transparent models. The evaluation further shows that significantly better results are obtained when using the oracle-classified production data together with the original training data, instead of using only oracle data. An analysis of the fidelity of the transparent models to the oracles shows that performance gains can be expected from increasing oracle performance rather than from increasing fidelity. Finally, it is shown that further performance gains can be achieved by adjusting the relative weights of training data and oracle data.

Place, publisher, year, edition, pages
2012. Vol. 16, no 2, 247-263 p.
Keyword [en]
Classification, comprehensibility, decision trees, decision lists, oracle coaching
National Category
Computer and Information Science
URN: urn:nbn:se:su:diva-76744DOI: 10.3233/IDA-2012-0522ISI: 000301366100007OAI: diva2:527207


Available from: 2012-05-18 Created: 2012-05-16 Last updated: 2013-01-09Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full text

Search in DiVA

By author/editor
Boström, Henrik
By organisation
Department of Computer and Systems Sciences
In the same journal
Intelligent Data Analysis
Computer and Information Science

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 18 hits
ReferencesLink to record
Permanent link

Direct link