Change search
ReferencesLink to record
Permanent link

Direct link
An unexpected influence of widely used significance thresholds on the distribution of reported P-values
Uppsala universitet, Institutionen för ekologi och evolution.ORCID iD: 0000-0001-5791-336X
2007 (English)In: Journal of Evolutionary Biology, ISSN 1010-061X, E-ISSN 1420-9101, Vol. 20, no 3, 1082-1089 p.Article in journal (Refereed) Published
Abstract [en]

We consider the problematic relationship between publication success and statistical significance in the light of analyses in which we examine the distribution of published probability (P) values across the statistical `significance' range, below the 5% probability threshold. P-values are often judged according to whether they lie beneath traditionally accepted thresholds (< 0.05, < 0.01, < 0.001, < 0.0001); we examine how these thresholds influence the distribution of reported absolute P-values in published scientific papers, the majority in biological sciences. We collected published P-values from three leading journals, and summarized their distribution using the frequencies falling across and within these four threshold values between 0.05 and 0. These published frequencies were then fitted to three complementary null models which allowed us to predict the expected proportions of P-values in the top and bottom half of each inter-threshold interval (i.e. those lying below, as opposed to above, each P-value threshold). Statistical comparison of these predicted proportions, against those actually observed, provides the first empirical evidence for a remarkable excess of probability values being cited on, or just below, each threshold relative to the smoothed theoretical distributions. The pattern is consistent across thresholds and journals, and for whichever theoretical approach used to generate our expected proportions. We discuss this novel finding and its implications for solving the problems of publication bias and selective reporting in evolutionary biology.

Place, publisher, year, edition, pages
2007. Vol. 20, no 3, 1082-1089 p.
Keyword [en]
probability, significance, statistical analysis, statistics
National Category
Biological Sciences
URN: urn:nbn:se:su:diva-116496DOI: 10.1111/j.1420-9101.2006.01291.xISI: 000245989600025PubMedID: 17465918OAI: diva2:806562
Available from: 2008-01-22 Created: 2015-04-20 Last updated: 2015-04-20

Open Access in DiVA

No full text

Other links

Publisher's full textPubMed

Search in DiVA

By author/editor
Kolm, Niclas
In the same journal
Journal of Evolutionary Biology
Biological Sciences

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 46 hits
ReferencesLink to record
Permanent link

Direct link