Inter-Rater Agreement & QA

Measure and improve reviewer consistency. Learn inter-rater agreement metrics (Cohen's kappa, Krippendorff's alpha, percent agreement vs chance-corrected), QA sampling strategies, calibration sessions, the link from low IRA to ambiguous policy, bias diagnostics that catch reviewer-pool skew, and the operational practice that keeps IRA above floor without burning reviewers.