How AI is applied across API Evangelist and APIs.io. Read my AI disclosure →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

RCP-nDCG recalibrates retrieval evaluation with rubric-anchored preferences

calendar_today October 3, 2026 person Nils Reimers domain aligned-news

Rubric-Calibrated Preferences combine listwise comparisons with shared yes-or-no relevance criteria and item-response theory. In a blind study, RCP-nDCG agreed with human-preferred rerankers substantially more often than conventional nDCG when the metrics disagreed.

open_in_new Read original post