Predicting Replicability Challenge: Round 3 Results

COS launched a public competition in 2025 to test automated methods for predicting whether a research claim would be successfully replicated in a new sample of data. Through three rounds, participating teams were provided training data from existing replication studies along with metadata about a set of original claims. Teams were responsible for generating 0-1 confidence scores that measured the likelihood of successful replication, which were evaluated against the corresponding outcomes of replication attempts conducted on those claims.