An Introduction to Z-Curve: A Method to Evaluate the Credibility of Published Results Selected for Significance

It is well known that published articles in psychology journals are selected to present statistically significant results. Lately, it has become evident that this selection process can result in large numbers of replication failures. Z-Curve is a selection model that makes it possible to estimate the success rate of exact replication studies with the same sample size. Z-Curve 2.0 also makes it possible to quantify the extent of selection bias by comparing the observed discovery rate (i.e., the percentage of significant results) to the expected discovery rate (i.e., the unconditional average power before selection for significance). Finally, z-curve 2.0 provides an estimate of the false discovery risk. Based on this estimate it is possible to adjust alpha post-hoc to achieve an acceptable false discovery risk for a set of published results. I illustrate z-curve with applications to medical research, psychological research, psychology journals, and psychology departments.

To prepare for the seminar, please check out February 18, 2022 Z-Curve Webinar