Pueblo Serna

In celebration of the third anniversary of this blog, I'm collecting some of my posts on reproducibility. I didn't initially anticipate that methods and the "reproducibility crisis" in psychology would be my primary blogging topic, but it's become a huge part of what I write about on a day-to-day basis. Here are my top four posts in this sequence: A moderate's view of the reproducibility crisis  � part 1 of a sequence, in part responding to the release of the Open Science Collaboration reproducibility project paper .  The slower, harder ways to increase reproducibility  � part 2 of the sequence. Estimating p(replication) in a practical setting  � a report on the results from my graduate methods course, in which students replicate previously published papers.  Shifting our cultural understanding of replication  � a plea for changes in practices and incentives. Then I've also written substantially about a number of other topics, including publication incenti...

(tl;dr: I argue that the only way to tell if a replication study was successful is by considering the theory that motivated the original.) Psychology is in the middle of a sea change in its attitudes towards direct replication. Despite their value in providing evidence for the reliability of a particular experimental finding, incentives for direct replications have typically been limited . Increasingly, however, journals and funding agencies  now increasingly value these sorts of efforts. One major challenge, however, has been evaluating the success of direct replications studies. In short, how do we know if the finding is the same? There has been limited consensus on this issue, so many projects have used a diversity of methods. The RP:P 100-study replication project , reports several indicators of replication success, including 1) the statistical significance of the replication, 2) whether the original effect size lies within the confidence interval of the replication, 3) the re...