Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition
Stockholm University, Faculty of Social Sciences, Stress Research Institute. Stanford University, USA; Karolinska Institutet, Sweden.ORCID iD: 0000-0001-5273-0150
Show others and affiliations
Number of Authors: 142018 (English)In: Royal Society Open Science, E-ISSN 2054-5703, Vol. 5, no 8, article id 180448Article in journal (Refereed) Published
Abstract [en]

Access to data is a critical feature of an efficient, progressive and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data ('analytic reproducibility'). To investigate this, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition. Interrupted time-series analyses indicated a substantial post-policy increase in data available statements (104/417, 25% pre-policy to 136/ 174, 78% post-policy), although not all data appeared reusable (23/ 104, 22% pre-policy to 85/136, 62%, post-policy). For 35 of the articles determined to have reusable data, we attempted to reproduce 1324 target values. Ultimately, 64 values could not be reproduced within a 10% margin of error. For 22 articles all target values were reproduced, but 11 of these required author assistance. For 13 articles at least one value could not be reproduced despite author assistance. Importantly, there were no clear indications that original conclusions were seriously impacted. Mandatory open data policies can increase the frequency and quality of data sharing. However, suboptimal data curation, unclear analysis specification and reporting errors can impede analytic reproducibility, undermining the utility of data sharing and the credibility of scientific findings.

Place, publisher, year, edition, pages
2018. Vol. 5, no 8, article id 180448
Keywords [en]
open data, reproducibility, open science, meta-science, interrupted time series, journal policy
National Category
Biological Sciences Public Administration Studies Media and Communications Computer and Information Sciences
Identifiers
URN: urn:nbn:se:su:diva-160133DOI: 10.1098/rsos.180448ISI: 000443443000059OAI: oai:DiVA.org:su-160133DiVA, id: diva2:1249453
Available from: 2018-09-19 Created: 2018-09-19 Last updated: 2018-09-19Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full text

Search in DiVA

By author/editor
Nilsonne, Gustav
By organisation
Stress Research Institute
In the same journal
Royal Society Open Science
Biological SciencesPublic Administration StudiesMedia and CommunicationsComputer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 44 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf