Endre søk
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
On the Calibration of Aggregated Conformal Predictors
Dept. of Information Technology, University of Borås, Sweden.
Stockholms universitet, Samhällsvetenskapliga fakulteten, Institutionen för data- och systemvetenskap. Swetox, Karolinska Institutet, Unit of Toxicology, Sweden.
Stockholms universitet, Samhällsvetenskapliga fakulteten, Institutionen för data- och systemvetenskap.
Dept. of Computer Science and Informatics, Jönköping University, Sweden; Dept. of Information Technology, University of Borås, Sweden.
Vise andre og tillknytning
2017 (engelsk)Inngår i: Proceedings of Machine Learning Research: Volume 60: Conformal and Probabilistic Prediction and Applications, 13-16 June 2017, Stockholm, Sweden / [ed] Alex Gammerman, Vladimir Vovk, Zhiyuan Luo, Harris Papadopoulos, 2017, Vol. 60, s. 154-173Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Conformal prediction is a learning framework that produces models that associate with each of their predictions a measure of statistically valid confidence. These models are typically constructed on top of traditional machine learning algorithms. An important result of conformal prediction theory is that the models produced are provably valid under relatively weak assumptions—in particular, their validity is independent of the specific underlying learning algorithm on which they are based. Since validity is automatic, much research on conformal predictors has been focused on improving their informational and computational efficiency. As part of the efforts in constructing efficient conformal predictors, aggregated conformal predictors were developed, drawing inspiration from the field of classification and regression ensembles. Unlike early definitions of conformal prediction procedures, the validity of aggregated conformal predictors is not fully understood—while it has been shown that they might attain empirical exact validity under certain circumstances, their theoretical validity is conditional on additional assumptions that require further clarification. In this paper, we show why validity is not automatic for aggregated conformal predictors, and provide a revised definition of aggregated conformal predictors that gains approximate validity conditional on properties of the underlying learning algorithm.

sted, utgiver, år, opplag, sider
2017. Vol. 60, s. 154-173
Serie
Proceedings of Machine Learning Research, ISSN 2640-3498 ; 60
HSV kategori
Forskningsprogram
datalogi
Identifikatorer
URN: urn:nbn:se:su:diva-192606OAI: oai:DiVA.org:su-192606DiVA, id: diva2:1547114
Konferanse
Conformal and Probabilistic Prediction and ApplicationsVolume 60: Conformal and Probabilistic Prediction and Applications, 13-16 June 2017, Stockholm, Sweden
Tilgjengelig fra: 2021-04-25 Laget: 2021-04-25 Sist oppdatert: 2022-02-25bibliografisk kontrollert
Inngår i avhandling
1. Nonconformity Measures and Ensemble Strategies: An Analysis of Conformal Predictor Efficiency and Validity
Åpne denne publikasjonen i ny fane eller vindu >>Nonconformity Measures and Ensemble Strategies: An Analysis of Conformal Predictor Efficiency and Validity
2021 (engelsk)Doktoravhandling, med artikler (Annet vitenskapelig)
Abstract [en]

Conformal predictors are a family of predictive models that associate with each of their predictions a measure of confidence, enabling them to provide quantitative information about their own trustworthiness. In risk-laden machine learning applications, where bad predictions may lead to economic loss, personal injury, or worse, such inherent quality control appears highly beneficial, if not required. While the foundations of conformal prediction were initially published some twenty years ago, their use, and further development, is still (at the time of writing this thesis) not widespread in the machine learning community, and several open questions remain regarding the proper design and use of conformal prediction systems. In this thesis, we attempt to tackle some of these questions, focusing our attention on three specific characteristics of conformal predictors. First, conformal predictors rely on so-called nonconformity functions, which are mappings from the object space onto the real line, typically based on traditional classification or regression models; here, we investigate properties of the underlying learning algorithm and characteristics of the resulting conformal predictor. Second, conformal predictors output predictions on a form that is distinct from traditional prediction methods, by supplying multi-valued prediction regions with a statistically valid coverage probability; we propose two procedures for post-processing the output from conformal classification models that provide interpretations more closely related to traditional predictive models, while still retaining meaningful confidence information. Finally, we provide contributions relating to the construction of conformal predictor ensembles, illustrating potential issues with existing ensemble procedures, as well as proposing and evaluating an alternative ensemble method.

Abstract [sv]

Avhandlingen behandlar områdetconformal prediction, som beskriver en fa-milj prediktiva modeller vars prediktioner associeras med ett konfidensmått,som låter modellerna själva uttrycka sig om sin egen tillförlitlighet. I hög-riskapplikationer, där dåliga prediktioner kan få allvarliga ekonomiska konse-kvenser, eller leda till personskada, tycks en sådan inbyggd säkerhetskontrollhögst värdefull, om inte nödvändig. Medan den teoretiska grunden till confor-mal prediction lades för cirka 20 år sedan, är forskningsområdet fortfaranderelativt ungt, och många öppna frågor kvarstår gällande design och använd-ning av conformal prediction-system. I avhandlingen behandlas några av des-sa öppna frågor, och fokus läggs på tre specifika karakteristika hos conformal-prediktorer. Först behandlas de så kallade icke-konformitetsfunktionerna (non-conformity functions) som ligger till grund för conformal prediction, och sam-bandet utforskas mellan egenskaper hos icke-konformitetsfunktionerna och deresulterande prediktorerna. även egenskaper hos de prediktioner som produ-ceras i en conformal predictor undersöks, och två post-processeringsmetoderpresenteras i ett försök att bistå med en mer intuitivt begriplig tolkning av des-sa prediktioner. Slutligen utforskas strategier för konstruktion av ensemblerav conformal prediction-modeller, där svagheter illustreras i vedertagna stra-tegier, följt av en presentation av en ny ensemblestrategi som ämnar adresseradessa svagheter.

sted, utgiver, år, opplag, sider
Stockholm: Department of Computer and Systems Sciences, Stockholm University, 2021. s. 62
Serie
Report Series / Department of Computer & Systems Sciences, ISSN 1101-8526 ; 21-001
Emneord
Data Science, Machine Learning, Conformal Prediction, Classification, Regression
HSV kategori
Forskningsprogram
data- och systemvetenskap
Identifikatorer
urn:nbn:se:su:diva-192613 (URN)978-91-7911-502-9 (ISBN)978-91-7911-503-6 (ISBN)
Disputas
2021-06-14, online via Zoom, public link is available at the department website, Stockholm, 13:00 (engelsk)
Opponent
Veileder
Tilgjengelig fra: 2021-05-20 Laget: 2021-04-25 Sist oppdatert: 2022-02-25bibliografisk kontrollert

Open Access i DiVA

Fulltekst mangler i DiVA

Andre lenker

Fri fulltext

Person

Norinder, UlfBoström, Henrik

Søk i DiVA

Av forfatter/redaktør
Norinder, UlfBoström, Henrik
Av organisasjonen

Søk utenfor DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric

urn-nbn
Totalt: 150 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf