esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

AI engineering / Advanced / 03.21

A model as the judge

How one model can grade another model's answers with a rubric, how it gets fooled, and how to check the judge against people.

14 min readFree

Using a model to grade a model

The last lesson, Deterministic Evals, ended with a promise: next, model-based evals, using a model to grade another model. Here it is.

Deterministic checks are rules. Does the answer contain the order number? Is the reply valid JSON? A rule gives the same verdict every time, and it costs almost nothing. But a rule cannot tell you whether an answer was kind, or whether it stuck to the facts you gave the model. Those are the things a person would judge by reading.

So teams ask a second model to read the answer and grade it. Think of a teacher marking essays with a printed marking sheet. The student is the model that wrote the answer. The teacher is the judge. The marking sheet is the rubric. In this lesson you will build the marking sheet, mark six answers, and then check whether the teacher is any good.

Key words

Eval
A repeatable test of how good a model's answers are, run on a fixed set of examples.
Judge
A model whose job is to grade another model's answers. People also call this LLM-as-a-judge.
Rubric
Written grading rules: what each score from 1 to 5 means, in plain words.
Pointwise
Grading one answer at a time, giving it a score on its own.
Pairwise
Showing the judge two answers and asking which one is better.
Bias
A habit that pushes the judge towards an answer for a reason that has nothing to do with quality.

What the research says

The best-known study of this idea is a 2023 paper on arXiv called "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena" (2306.05685). Its authors start from a problem: chat assistants are hard to test, because ordinary benchmarks do not measure what people actually prefer. So they try strong models as judges on open-ended questions.

The abstract says they "examine the usage and limitations of LLM-as-a-judge, including position, verbosity, and self-enhancement biases, as well as limited reasoning ability", and that they propose solutions to mitigate some of them. In plain words: the judge is useful, but it has habits that can fool you.

The good news in the abstract is about agreement. The authors report that strong model judges matched human preferences well, with over 80% agreement, which they say is the same level as the agreement between two humans. They call LLM-as-a-judge a scalable and explainable way to approximate human preferences, which are otherwise very expensive to obtain.

Pointwise or pairwise

There are two common ways to ask a judge for its opinion.

  • Pointwise

    Give the judge one answer and the rubric. It returns a score, such as 4 out of 5. Good for tracking quality over time, because every answer gets a number.

  • Pairwise

    Give the judge two answers to the same question and ask which is better. Good for comparing two versions of a prompt, because choosing is easier than scoring.

  • What you grade

    The rubric names the quality: helpfulness (did it solve the problem), faithfulness to sources (did it stay inside the facts you gave), or tone (does it sound right for your brand). One rubric per quality works best.

How a judge gets fooled

The paper names the habits above. Here is what each one looks like in daily life. These explanations are ours, not the paper's wording.

  • Position bias

    Leaning on where an answer sits, such as liking whichever one is shown first. Like a tired tasting panel that always votes for the first cake.

  • Verbosity bias

    Liking the longer answer because it is longer, even when the extra words add nothing. Like marking an essay by weight.

  • Self-enhancement bias

    Favouring answers that look like the judge's own writing style, such as answers from the same model. Like a chef who prefers their own cooking.

The fourth item in the abstract is "limited reasoning ability". A judge can slip on maths or logic that a careful person would catch. If your answers hinge on a calculation, add a rule-based check for that part, as the Deterministic Evals lesson showed.

Making a judge you can trust

A judge is just another model, so its grades are only useful if you test it. Here is a short recipe.

  1. 1

    Write a clear rubric, with examples

    Say what each score means. Add one or two sample answers with their scores, so the judge sees where the lines are.

  2. 2

    Ask for the reason first, then the score

    Have the judge write one sentence of reasoning before it gives the number. You can read the reasons later to see why a grade looks odd.

  3. 3

    Ask for a fixed shape

    Tell it to reply with JSON such as {"reason": "...", "score": 3}. Then your program can read the score without guessing. Check that the reply really parses, because a model can break the format.

  4. 4

    Check it against humans on a sample

    Have a person grade a small set, even 20 answers. Compare the judge's grades with theirs. If they disagree a lot, fix the rubric before you trust the judge on thousands of answers.

  5. 5

    Swap positions in pairwise tests

    Run each pair twice, once as A then B and once as B then A. If the winner changes when only the order changed, you have caught position bias.

Step 4 is the same idea statisticians call inter-rater reliability. Wikipedia describes it as the "degree of agreement among independent observers" who rate the same thing, and says that assessment tools that rely on ratings must exhibit good inter-rater reliability, otherwise they are not valid tests. Here, the judge is one rater and the person is the other.

Build a judge prompt

Let us make the marking sheet. This program only builds the text that you would send to a judge. It does not call any model, so it runs free and needs no keys. The rubric grades faithfulness: does the answer stick to the source?

Python
RUBRIC = """Score the ANSWER from 1 to 5 for faithfulness to the SOURCE.
5 = every claim is in the source and it answers the question
4 = correct, but misses a small detail
3 = partly right, or mixes in one claim not in the source
2 = mostly not supported by the source
1 = wrong, invented, or off topic"""


def build_judge_prompt(question, source, answer):
    return (
        "You are a strict grader.\n\n"
        + RUBRIC
        + "\n\nSOURCE: " + source
        + "\nQUESTION: " + question
        + "\nANSWER: " + answer
        + "\n\nFirst write one sentence giving your reason."
        + '\nThen reply with JSON like {"reason": "...", "score": 3}.'
    )


source = "Returns are accepted for 30 days with a receipt."
print(build_judge_prompt("How long can I return an item?", source, "You have 30 days, with a receipt."))

Output

You are a strict grader.

Score the ANSWER from 1 to 5 for faithfulness to the SOURCE.
5 = every claim is in the source and it answers the question
4 = correct, but misses a small detail
3 = partly right, or mixes in one claim not in the source
2 = mostly not supported by the source
1 = wrong, invented, or off topic

SOURCE: Returns are accepted for 30 days with a receipt.
QUESTION: How long can I return an item?
ANSWER: You have 30 days, with a receipt.

First write one sentence giving your reason.
Then reply with JSON like {"reason": "...", "score": 3}.

Look at what the prompt does. It gives the score meanings, shows the source, the question and the answer, asks for one sentence of reasoning first, and asks for JSON. That is steps 1 to 3 of the recipe in one function.

Tip

Keep the source, question and answer clearly labelled. A judge that has to guess where the answer starts will grade the wrong text.

Score six answers and check the judge

Now we need a judge. A real one is a model call: slower than a normal function, it costs money for every token, it can give a different grade on each run, and it can fail or refuse. We will not call one here.

Instead, the next program uses a stand-in judge, a plain rule that we wrote so the code runs free. It is not a real model and it does not understand meaning. It simply counts how many of the answer's words appear in the source. It is deliberately a weak judge, so you can see what a check against humans reveals.

The question is "How long do I have to return an item?". We have six answers, and six human grades that a person gave after reading each one.

Python
import re

SOURCE = ("Our shop ships orders within 2 business days. Returns are accepted "
          "for 30 days with a receipt. Delivery to Canada takes 5 to 7 days.")
QUESTION = "How long do I have to return an item?"

ANSWERS = [
    "Returns are accepted for 30 days with a receipt.",
    "You can return an item within 30 days if you keep the receipt.",
    "Returns are accepted for 60 days with a receipt.",
    "Delivery to Canada takes 5 to 7 days.",
    "You can return items for 30 days. Refunds arrive in 3 days by bank transfer.",
    "Our shop ships orders within 2 business days.",
]
HUMAN = [5, 5, 2, 1, 3, 1]  # six scores written by a person who read each answer


def words(text):
    return [w for w in re.findall(r"[a-z0-9]+", text.lower()) if len(w) > 3 or w.isdigit()]


def fake_judge(question, source, answer):
    # STAND-IN: no model here. It counts how many answer words appear in the source.
    # It cannot understand meaning, so a wrong number such as "60 days" slips past it.
    mine = words(answer)
    seen = set(words(source))
    share = sum(w in seen for w in mine) / len(mine)
    score = 1 + round(share * 4)
    reason = f"{round(share * 100)}% of the answer's words appear in the source"
    return {"reason": reason, "score": score}


judge = [fake_judge(QUESTION, SOURCE, a) for a in ANSWERS]
for i, (j, h) in enumerate(zip(judge, HUMAN), 1):
    print(f"answer {i}: judge={j['score']} human={h}  ({j['reason']})")

exact = sum(j["score"] == h for j, h in zip(judge, HUMAN))
near = sum(abs(j["score"] - h) <= 1 for j, h in zip(judge, HUMAN))
print(f"exact agreement: {exact}/6")
print(f"within 1 point:  {near}/6")

Output

answer 1: judge=5 human=5  (100% of the answer's words appear in the source)
answer 2: judge=3 human=5  (57% of the answer's words appear in the source)
answer 3: judge=4 human=2  (83% of the answer's words appear in the source)
answer 4: judge=5 human=1  (100% of the answer's words appear in the source)
answer 5: judge=2 human=3  (30% of the answer's words appear in the source)
answer 6: judge=5 human=1  (100% of the answer's words appear in the source)
exact agreement: 1/6
within 1 point:  2/6

Two kinds of agreement are printed. Exact agreement counts the answers where judge and human gave the same score. Within-1 agreement counts the answers where the scores are at most one point apart, which is gentler, since the difference between a 4 and a 5 is often a matter of taste.

agreement=answers where judge and human matchall answers\text{agreement} = \frac{\text{answers where judge and human match}}{\text{all answers}}

In words: count the matches and divide by the number of answers. Our stand-in matched exactly on 1 of 6 answers and was within one point on 2 of 6. That is poor, and the reasons are clear in the output.

Answers 4 and 6 are true sentences from the source, but they do not answer the question. The stand-in gave them 5, the human gave 1. Answer 3 says 60 days where the source says 30, and the stand-in still gave it 4. A word counter cannot see an off-topic answer or one wrong number.

This is the point of the human sample. A weak judge would have quietly graded thousands of answers wrongly. The check told us not to trust it.

Quick check

Your judge agrees with human grades on only 2 of 6 sample answers. What should you do?

Catch position bias by swapping

Now the pairwise case. Here the stand-in judge has two bad habits built in on purpose, so that we can catch them. A clear length gap wins (verbosity bias), and a near tie always goes to the first answer (position bias). Again, this is a stand-in rule, not a real model.

For each pair we run the test twice. In run one, x is shown first. In run two, y is shown first. A fair judge picks the same answer both times.

Python
def biased_judge(first, second):
    # STAND-IN with two bad habits on purpose, so we can catch them:
    # a clear length gap wins (verbosity), a near tie always goes to the first one (position).
    if abs(len(first) - len(second)) > 15:
        return "first" if len(first) > len(second) else "second"
    return "first"


PAIRS = [
    ("You can return items for 30 days with a receipt.", "Returns are accepted for 30 days with a receipt."),
    ("Delivery to Canada takes 5 to 7 days.", "Returns are accepted for 30 days with a receipt."),
    ("Returns are accepted for 30 days with a receipt.",
     "Returns are accepted for 30 days with a receipt, and we are always happy to help with anything else you need."),
    ("Keep your receipt and return it within 30 days.", "Return it in 30 days with the receipt."),
]

stable = 0
for n, (x, y) in enumerate(PAIRS, 1):
    run1 = x if biased_judge(x, y) == "first" else y   # x shown first
    run2 = y if biased_judge(y, x) == "first" else x   # y shown first
    same = run1 == run2
    stable += same
    print(f"pair {n}: x first -> {'x' if run1 == x else 'y'}, y first -> {'x' if run2 == x else 'y'}, "
          f"{'consistent' if same else 'FLIPPED: position bias'}")
print(f"consistent in {stable} of {len(PAIRS)} pairs")

Output

pair 1: x first -> x, y first -> y, FLIPPED: position bias
pair 2: x first -> x, y first -> y, FLIPPED: position bias
pair 3: x first -> y, y first -> y, consistent
pair 4: x first -> x, y first -> y, FLIPPED: position bias
consistent in 1 of 4 pairs

Three pairs flipped. In pair 1 the two answers are about equally good, yet the winner was whichever one came first. That is position bias, caught without any human, just by swapping the order.

Pair 3 is consistent, but look at why. The second answer in that pair is padded with a friendly sentence that adds nothing, and the judge chose it both times. That is verbosity bias, and swapping cannot catch it, because the padded answer wins in both orders. To catch that one, add a rubric line such as "do not prefer an answer for being longer", and compare against human picks.

Tip

Treat a flipped pair as a tie, not as a win for either side. Count how many pairs flip. If many do, the judge cannot separate your two versions.

What judging costs

Every grade is a model call, so judging has a price in money and in time. Here is a sum with made-up numbers, only to show the shape. Pretend one judge call costs 0.002 dollars. This number is invented; real prices change and differ between models.

  1. 1

    One pass

    1,000 answers, one pointwise grade each: 1,000 calls, which is 2 dollars at the made-up price.

  2. 2

    Pairwise with swaps

    Each pair is judged twice, so the same 1,000 comparisons cost 2,000 calls, or 4 dollars.

  3. 3

    Run it after every change

    Ten prompt changes, each re-judged: 40 dollars. It grows fast with long answers, because the judge reads everything every time.

Ways to keep it sensible: judge a sample instead of everything, grade with cheap rules first and send only the unclear answers to the judge, and keep the judge prompt short. A real judge is slower than a rule and can grade differently on a second run, so re-check it against your human set whenever you change it.

Recap

A model can grade another model when you need judgement that rules cannot give. Write a rubric with examples, ask for the reason before the score, compare the judge with human grades on a sample, and swap positions in pairwise tests.

Quick check

In a pairwise test, the winner changes when you only swap the order of the two answers. What does that show?

Next: human evals, when people must judge the answers.

Sources

How is this lesson?

Last updated on 16 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.