Judge-human agreement measures how closely a judge model's scores match those of people on the same cases, using an agreement rate or a coefficient such as Cohen's kappa. High agreement justifies relying on the judge; otherwise the rubric or the judge needs to be reviewed.
Example
The judge and the human give the same score on 85 cases out of 100: raw agreement of 85%.