Did It Actually Work? — Evaluation and Failure Diagnosis
Lesson 9: Did It Actually Work? — Evaluation and Failure Diagnosis
A fine-tune without evaluation is a rumour. The model feels better, the examples you happened to try looked good, and nobody knows whether the improvement is real or whether you quietly broke three things you weren't looking at. This lesson is the discipline that separates a shipped model from a lucky one.
Measure the baseline first — before you train anything. Run your untouched base model on your test set and record the numbers. If you skip this, you have no way to know whether your fine-tune helped, did nothing, or made things worse. This is the single most-skipped step in the field, and it is not recoverable after the fact.
What to measure
Pick metrics that map to your actual failure mode, not a generic leaderboard score:
| Metric | What it catches | How to compute it |
|---|---|---|
| Format compliance rate | Whether the output parses and matches the schema you taught | Try to parse every test output. Report % success. This is the metric most fine-tunes are actually built for. |
| Exact match / accuracy | Correctness on tasks with a single right answer (classification, extraction, maths) | Programmatic comparison against the test labels. Fully deterministic, no judge needed. |
| Task-specific metric | Whatever your domain already measures (F1 on extraction, BLEU/ROUGE on translation, pass@k on code) | Reuse your existing evaluation code. |
| LLM-as-judge win rate | Open-ended quality where "correct" is fuzzy | Pairwise comparison: show a judge model the base output and the tuned output for the same prompt, ask which is better, then repeat with the order swapped and average out the position bias. |
| Human review | Everything the judge misses — tone, safety, subtle wrongness | 50–100 test outputs read by a person. Expensive and irreplaceable. Do it before shipping. |
| Regression suite | Capabilities you broke while fixing your task | A handful of unrelated prompts (reasoning, safety, general chat) run before and after. Non-negotiable for production. |
The pairing rule: a fine-tune claim needs two numbers, not one. "Format compliance went from 71% (base) to 98% (tuned) on a 300-example held-out test set." A single number is a marketing slide; a before/after pair is evidence.
Reading the loss curves
A rising loss on step 1 is not a normal warmup artefact. If loss climbs from the start, kill the run. The usual culprits are a learning rate 100× too high for the method, or a data pipeline that is training on the wrong tokens (masking inverted, so the model is being taught to generate the user's message).
Symptom → cause → fix
| Symptom | Likely cause | Fix |
|---|---|---|
| Great on training data, poor on anything new | Overfitting — too many epochs, rank too high, or synthetic near-duplicates | Cut epochs to 1–2, reduce rank, dedupe harder. Add more diverse data. |
| Trained model is worse at everything than the base | Catastrophic forgetting — learning rate too high or too many epochs on narrow data | Lower LR (try 1e-4 for LoRA), 1 epoch, and add 10–20% general-purpose examples to the mix. |
| Always produces the same answer | Mode collapse — data too homogeneous, or the task is dominated by one majority class | Rebalance the dataset; add variety in length, phrasing and outputs. |
| Correct content, wrong scaffolding / ignores system prompt | Chat-template mismatch between training and inference | Apply the model's own template function on both sides. Verify by decoding a training example back to text. |
| Outputs are much longer than before | Length imbalance — training answers are all long | Match the real output-length distribution. Add EOS/stop handling. |
| Refuses things it used to answer | Safety behaviour drifted, or refusal examples leaked into your dataset | Add the desired behaviour explicitly; run your regression suite. |
| Format compliance improved but accuracy dropped | You optimised for the easy half of the task | The dataset probably contains many well-formed wrong answers. Audit the training labels, not the model. |
| Nothing changed at all from the base model | Adapters not actually loaded, LR effectively zero, or wrong base checkpoint | Check the adapter file exists and the config reports non-zero trainable parameters before training. |
LLM-as-judge, done properly
1. PAIRWISE, not absolute. Ask "which of these two answers is better
for the request?" — never "rate this 1-10". Absolute scores drift
between runs and are not comparable.
2. SWAP THE ORDER. Run every pair twice (A-then-B, B-then-A) and
average. Judges have a measurable position bias toward whichever
answer comes first.
3. GIVE A RUBRIC, not a vibe. State the criteria: format compliance,
factual accuracy, tone. Unrubricked judges reward verbosity.
4. CALIBRATE ON HUMANS. Hand-label 30-50 pairs yourself and check the
judge agrees. An uncalibrated judge is a random-number generator
with good grammar.
5. TRACK THE JUDGE VERSION. When the judge model changes, your metric
changes. Record the model and the prompt alongside the score.
🪙 Token angle: running an LLM judge costs tokens, and it is easy to accidentally budget more for evaluation than for training. Generate the model outputs once, save them to a JSONL file, and score that fixed file with as many judge prompts as you like — instead of re-generating outputs on every eval iteration. Also cap judge input: for format-compliance checks, a deterministic parser costs nothing and is strictly better than a judge.
📺 Watch:
- Deep Dive: Fine-Tuning in Microsoft Foundry — SFT, DPO, Tool Calling and Cost — includes how managed platforms surface evaluation and cost per run.
- LLM Fine-Tuning Explained: The Complete Guide — covers baseline-vs-tuned comparison methodology.
🧠 Knowledge Check
1. What must you do before starting a fine-tune run?
2. Training loss keeps falling but validation loss is now rising. What is happening and what should you do?
3. Your fine-tune now emits perfect JSON but the answers inside it are frequently wrong. What's the most likely explanation?