LM7-T5 — Evaluating Instruction Following and Data Contamination

Learning objectives

After this lesson, you should be able to:

• Define measurable criteria for instruction following and choose a scoring method for each.
• Build a held-out evaluation set and keep it separate from training and tuning decisions.
• Compute exact-match, normalised and constraint-based scores and a confidence interval for them.
• Detect overlap between evaluation and training data with n-grams and explain what contamination does to a score.

The idea: decide what “correct” means before looking

Start from the use. For a robot assistant, “follows the instruction” may mean: produces exactly the expected command; uses only allowed actions; respects units and ranges; refuses what it should refuse; stops after the answer. Each can be written as a check. Fluent prose is not on the list.

Scoring methods

• Exact match against a reference: right when the answer is short and unique (a number, a label, a command). Normalise case, whitespace and punctuation first, or you will mark “Stopped.” wrong against “stopped”.
• Constraint checks: programmatic tests that the answer is valid JSON, contains the required fields, stays under a length, uses a listed action, includes or avoids specific words. Cheap and objective; test format and rules, not quality.
• Reference-based similarity (overlap or embedding similarity with a reference answer): fast but crude for free text.
• Rubric scoring by people: graders score each answer against a written rubric (correct, complete, safe, well formatted). The gold standard for open-ended answers; costly, and needs several graders and an agreement check (Lesson LM6-T4).
• Model-based judging: a strong model scores or compares answers with a rubric. Scales cheaply and correlates with human judgement on many tasks, with known biases (it can favour longer or its own style, and be inconsistent). Validate it against human scores on a sample before relying on it.

The held-out set

Build the evaluation set first, from instructions that represent real use, covering each kind of task and the edge cases (empty input, ambiguous request, request that must be refused). Keep it out of the training data, out of prompt tuning and out of every decision about which checkpoint to keep, or it becomes another training set. Report the number of items per category.

Contamination

Data contamination means that evaluation items, or their answers, appear in the training data. It can be exact (same text), near-duplicate (reworded or reformatted), or indirect (the benchmark was published and copied to many web pages that went into pretraining). A contaminated model can score high by recall rather than ability. The signs: scores that are much higher than on fresh items of the same type, performance that collapses when you rephrase the question, and verbatim reproduction of benchmark text. The remedies: deduplicate the training data against the evaluation set, build fresh or private evaluation items, and report how overlap was checked.

The mathematics: scores, uncertainty and overlap

Accuracy and its uncertainty. With k correct of n items, p̂ = k/n and the standard error is

For p̂ = 0.90 on n = 100 items: SE = 0.03 and the interval is 0.84 to 0.96. To distinguish two models whose true accuracies differ by 3 points you need on the order of a thousand items. Evaluation sets of 20 to 50 items per category show large differences only.

Paired comparison. Two models evaluated on the same items are better compared by looking at the items where they differ: n_A items only A gets right and n_B items only B gets right. If the models were equally good, each disagreement is a coin flip. With n_A = 12 and n_B = 3, the exact two-sided sign test gives a probability of about 0.035 for a split at least this uneven, so A is probably better; with 5 and 2 it would be about 0.45, so no conclusion.

N-gram overlap. Let G_n(x) be the set of word n-grams of a text x. The contamination score of an evaluation item e against a training set T is

G_n(x)

the set of all sequences of n consecutive words in x (n = 5 to 13 is typical; longer n means fewer false alarms).

G_n(T)

the union of n-gram sets over all training texts.

overlap(e)

between 0 (no shared n-gram) and 1 (every n-gram of e appears in training); a high value flags a likely leak.

Overlap detects exact and near-duplicate leaks but not paraphrases, which need embeddings or manual review.

Score inflation. If a fraction f of the evaluation items is leaked and the model answers leaked items correctly with probability a_leak and clean items with a_clean, the measured accuracy is f · a_leak + (1 − f) · a_clean. For a memorising model (a_leak = 1, a_clean = 0) and f = 0.5 this is 0.5, as in the figure.

A worked example: scoring, intervals and leak detection

The code scores four answers with strict and normalised exact match, checks constraints, computes an interval, counts disagreements for a paired comparison, and measures n-gram overlap between two evaluation items and a small training set.

import math
import re

def ngrams(text, n=5):
    w = text.lower().split()
    return {tuple(w[i:i + n]) for i in range(len(w) - n + 1)}

train = ["to convert centimetres to metres divide the number by one hundred and write the unit",
         "the robot picks up the red box and moves to the green bin"]
evalset = ["to convert centimetres to metres divide the number by one hundred please",
           "describe how a lidar measures distance using light"]

train_grams = set().union(*(ngrams(t) for t in train))
for e in evalset:
    g = ngrams(e)
    shared = len(g & train_grams)
    print(shared, len(g), round(shared / len(g), 2))        # 7 8 0.88   then   0 4 0.0   (the first item is a leak)

def strict(pred, ref):
    return pred == ref

def normalised(pred, ref):
    norm = lambda s: re.sub(r"[^a-z0-9. ]", "", s.lower().replace(".", "")).strip()
    return norm(pred) == norm(ref)

preds = ["2.5 metres", "Stopped.", "yes", "42"]
refs = ["2.5 metres", "stopped", "no", "42"]
print([strict(p, r) for p, r in zip(preds, refs)], [normalised(p, r) for p, r in zip(preds, refs)])
# [True, False, False, True] [True, True, False, True]

def meets_constraints(answer, required, forbidden=()):
    a = answer.lower()
    return all(r in a for r in required) and not any(f in a for f in forbidden)
print(meets_constraints("Turn left, then stop at the dock.", ["left", "stop"], ["jump"]), meets_constraints("Turn left.", ["left", "stop"]))   # True False

p, n = 0.9, 100
se = math.sqrt(p * (1 - p) / n)
print(round(se, 3), round(p - 1.96 * se, 2), round(p + 1.96 * se, 2))      # 0.03 0.84 0.96

a = [1, 1, 0, 1, 0, 1, 1, 1]
b = [1, 0, 0, 1, 0, 1, 0, 1]
only_a = sum(x == 1 and y == 0 for x, y in zip(a, b))
only_b = sum(x == 0 and y == 1 for x, y in zip(a, b))
print(only_a, only_b)                                                       # 2 0  (paired comparison: look at the disagreements)

The first item of the evaluation set shares 7 of its 8 five-grams with the training data: a near-duplicate that was reworded only at the end. The second shares none. In a real audit you would run this over every evaluation item and review those above a threshold.

A checklist for an honest evaluation

Engineering implications and limitations

Freeze the evaluation set and version it. Compare checkpoints and runs on exactly the same items; any change to the set breaks the comparison.

Decode deterministically for evaluation. Use greedy decoding or a fixed seed, so that a score difference is not sampling noise. If you sample, average over several seeds and report the spread (Lesson LM5-T5).

Score the format and the content separately. A correct answer in the wrong format fails a parser; a well-formatted wrong answer passes a format check. Report both.

Test refusals and edge cases. Include requests the model must refuse or clarify, empty inputs, very long inputs and adversarial phrasing. Instruction following includes knowing what not to do.

Use model judges carefully. Randomise the order of two answers being compared, use a written rubric, and measure agreement with humans. Do not use the model being evaluated as its own judge.

Keep a private test set. Public benchmarks leak. A small private set of items you wrote yourself, from real use, is worth more than a large public one for your decision.

Record how overlap was checked. State the n-gram size, the training data that were searched, and what was done with flagged items.

For robots. Evaluate in the loop: run the commands the model produces in a simulator or against a validator, and score outcomes (reached the goal, violated a constraint) rather than text similarity. Add tests for safety behaviours, and re-run them whenever the model, the prompt or the tools change.

Common misconceptions

Engineering takeaways

1. Define measurable criteria first: exact match with normalisation, constraint checks, rubric or model-based judging validated against humans.
2. Build the held-out set before training and keep it out of every decision.
3. Uncertainty: SE = √(p̂(1 − p̂)/n), about ±6 points at 90% on 100 items; compare models on the items where they disagree.
4. Contamination: exact, near-duplicate or indirect overlap of evaluation items with training data; detect with n-gram overlap, remedy with fresh or private items.
5. A leak inflates the score of a memorising model (0.0 clean versus 0.5 with half the items leaked), without any improvement in ability.

References and further reading

See Chapter 7 of the book (evaluating the finetuned LLM, including scoring responses with another model). The paired-comparison probabilities and overlap examples in this lesson can be reproduced with the code shown.

• Ouyang, L. et al. (2022). Training language models to follow instructions with human feedback. arXiv:2203.02155 (human evaluation of instruction following).
• Zheng, L. et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. arXiv:2306.05685.
• Build a Large Language Model (From Scratch), Chapter 7: Finetuning to Follow Instructions (primary book reference)

Knowledge check

Three conceptual questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

What’s next

That completes Module 7. Next, Module 8 adapts models efficiently with LoRA, quantisation and related methods.

Related content

Nothing below is required to finish this lesson.

• Before
  LM7-T4: Masking the Loss

  Compute loss only over the response tokens and handle padding with an ignore index.

• Lab
  LM7-L3: Instruction-Tune a Tiny Model and Evaluate It

  Fine-tune a small GPT on instruction-formatted examples and evaluate responses against predefined criteria.