Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Variability in the analysis of a single neuroimaging dataset by many teams
Show others and affiliations
Number of Authors: 1972020 (English)In: Nature, ISSN 0028-0836, E-ISSN 1476-4687, Vol. 582, p. 84-88Article in journal (Refereed) Published
Abstract [en]

Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses(1). The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset(2-5). Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed. The results obtained by seventy different teams analysing the same functional magnetic resonance imaging dataset show substantial variation, highlighting the influence of analytical choices and the importance of sharing workflows publicly and performing multiple analyses.

Place, publisher, year, edition, pages
2020. Vol. 582, p. 84-88
National Category
Neurosciences Psychology
Research subject
Psychology
Identifiers
URN: urn:nbn:se:su:diva-182960DOI: 10.1038/s41586-020-2314-9ISI: 000535225300001PubMedID: 32483374OAI: oai:DiVA.org:su-182960DiVA, id: diva2:1452995
Available from: 2020-07-08 Created: 2020-07-08 Last updated: 2022-02-26Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textPubMed

Authority records

Holzmeister, FelixCamerer, Colin F.Dreber, AnnaHuber, JuergenJohannesson, MagnusKirchler, MichaelAvesani, PaoloCecchetti, LucaDi, XinDukart, JuergenGau, RemiHawco, ColinHoffstaedter, FelixJoseph, Michael J. E.Losecaat Vermeer, Annabel B.Mack, Michael L.Nilsonne, GustavNotter, Michael P.Papale, PaoloPischedda, DorisSchultz, Douglas H.Smith, David V.Turella, Lucavan 't Veer, Anna E.White, David J.Wisniewski, DavidYoon, Sangsuk

Search in DiVA

By author/editor
Holzmeister, FelixCamerer, Colin F.Dreber, AnnaHuber, JuergenJohannesson, MagnusKirchler, MichaelAvesani, PaoloCecchetti, LucaDi, XinDukart, JuergenGau, RemiHawco, ColinHoffstaedter, FelixJoseph, Michael J. E.Losecaat Vermeer, Annabel B.Mack, Michael L.Nilsonne, GustavNotter, Michael P.Papale, PaoloPischedda, DorisSchultz, Douglas H.Smith, David V.Turella, Lucavan 't Veer, Anna E.White, David J.Wisniewski, DavidYoon, Sangsuk
By organisation
Biological psychology
In the same journal
Nature
NeurosciencesPsychology

Search outside of DiVA

GoogleGoogle Scholar

doi
pubmed
urn-nbn

Altmetric score

doi
pubmed
urn-nbn
Total: 261 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf