LM11-T1 — Base Versus Instruction-Tuned Models and Prompt Design

Learning objectives

After this lesson, you should be able to:

• Choose between a base and an instruction-tuned model for a task.
• Write a prompt with instruction, delimiters, examples and a specified output format.
• Request structured output and handle invalid replies with validation and bounded retries.
• Test and version prompts like code and state what prompting cannot guarantee.

The idea: ask the model the way it was trained to be asked

A base model (Module 5) was trained to continue text. Give it “The capital of France is” and it continues. Give it “Translate to French: cheese” and it may continue with another exercise, a list of words or a forum post: nothing in its training said that an instruction should be obeyed. An instruction-tuned model (Module 7) was further trained on pairs of instructions and good responses, so a request is treated as a request. A chat model additionally expects a conversation format with roles (system, user, assistant) and was often tuned on human preferences as well.

Which one do you call?

The anatomy of a prompt

A good prompt for an application has the same parts every time:

1. Role and task,One clear sentence saying what to do and for whom: “Classify each robot log line as positive or negative.”
2. Constraints,What not to do, what to do when unsure: “If you cannot tell, answer unknown.” A model told how to fail fails more predictably.
3. Delimited inputs,Put user text and documents between clear markers, and say that they are data. This is the first defence against injected instructions (Lesson LM11-T6).
4. Examples,Two to five input-output pairs that show the format and the hard cases. They teach the pattern better than a paragraph of description.
5. Output format,Say exactly what the reply must look like, ideally as a JSON object with named fields, and show one.

Decoding settings are part of the prompt

Temperature 0 (greedy decoding, Module 5) makes the output nearly deterministic, which is what you want for classification, extraction and tool calls. Higher temperature and nucleus sampling add variety, which is wanted for brainstorming and harmful for parsing. Set a maximum length so a runaway reply cannot cost you unbounded time or money. Even at temperature 0 results may vary between model versions and sometimes between runs on different hardware, so pin the model version.

The prompt is not a security boundary

The mathematics: why wording changes the output, and how to count it

A language model defines a distribution over the next token given everything before it (Module 1). A prompt x is nothing but the conditioning context, and the reply y is sampled from

Changing the wording changes x, therefore every factor, which is why small edits can change behaviour in ways nobody planned. Few-shot examples work because the model has seen many documents in which a pattern repeats; the examples place the continuation in that family. The instruction-tuned model has additionally been trained on x that look like instructions, so p(y | x) concentrates on responses.

Reliability of a structured reply

Suppose a single attempt yields a valid, schema-conforming reply with probability r, independently across attempts. With at most m tries:

r

per-attempt validity rate, measured on a test set of prompts, for example 0.9.

m

the retry limit, for example 3.

E[tries]

expected number of model calls (cost and latency multiply by this).

For r = 0.9 and m = 3: success 1 − 0.001 = 0.999; expected tries (1 − 0.001)/0.9 = 1.11. For r = 0.5: success 0.875, expected tries 1.75. The independence assumption is optimistic: a prompt that fails once tends to fail again on the same input, so measure the success rate of the whole loop, not only r. A rate of 99.9% still means one failure in a thousand requests, so the fallback path must exist and be tested.

Prompt length is a cost

Prefill cost grows with prompt length (Lesson LM10-T5) and API prices count prompt and reply tokens. Few-shot examples that are repeated in every call are therefore a recurring cost; trim them to the smallest set that keeps your test score.

A worked example: a prompt builder with a retry loop

The code builds a few-shot prompt, then runs a loop around a fake model that first answers with extra text around the JSON and then with a valid but extended object, to show the machinery of parse, validate and retry. Replace fake_model by a real call to use it with a model.

import json

def build_prompt(task, examples, query):
    parts = [task]
    for text, label in examples:
        parts.append(f"Input: {text}\nOutput: {label}")
    parts.append(f"Input: {query}\nOutput:")
    return "\n\n".join(parts)

prompt = build_prompt("Classify the sentiment of each robot log line as positive or negative.",
                      [("Arm reached target smoothly", "positive"), ("Motor stalled twice", "negative")],
                      "Gripper slipped on the box")
print(prompt)
print(len(prompt.split()))                     # 32 words

replies = iter(['Sure! Here is the JSON: {"label": "negative"}',
                '{"label": "negative", "confidence": 0.8}'])
def fake_model(prompt):
    return next(replies)

def ask(prompt, max_tries=3):
    for attempt in range(1, max_tries + 1):
        text = fake_model(prompt)
        try:
            obj = json.loads(text)                                  # parse
        except ValueError:
            prompt += "\nYour last reply was not valid JSON. Reply with JSON only."   # feed the error back
            continue
        ok = (set(obj) == {"label", "confidence"} and obj["label"] in ("positive", "negative")
              and isinstance(obj["confidence"], float))             # validate
        if ok:
            return obj, attempt
    return None, max_tries                                          # safe fallback, never a guess

print(ask(prompt))                              # ({'label': 'negative', 'confidence': 0.8}, 2)

for r, m in ((0.9, 3), (0.5, 3)):
    print(r, m, round(1 - (1 - r) ** m, 3), round((1 - (1 - r) ** m) / r, 2))
# 0.9 3 0.999 1.11
# 0.5 3 0.875 1.75

Note what the loop does not do: it never repairs a reply by guessing. A reply that does not validate is retried with the error, and after the last retry the application takes its fallback (ask the user, use a default, hand over to a human).

Treat prompts as code

Engineering implications and limitations

Specify the output before the prompt. Write the schema or the allowed labels first, then write a prompt that asks for them. The validator and the prompt then agree by construction.

Prefer constrained decoding where available. Some serving stacks restrict generation to tokens that keep the reply valid under a grammar or a schema. This raises the validity rate toward 100% but does not make the content correct: validate meaning and ranges separately.

Measure, then change. A prompt edit is a hypothesis. Run the test set before and after, report the difference with an interval, and keep the change only if it helps.

Keep examples representative and few. Examples bias the model toward their labels and their wording; if all examples are positive the model will lean positive. Balance them, and include a case of the failure mode you worry about.

Pin versions. Hosted models change. Pin the model identifier, log it, and re-run the test set when you upgrade.

Bound everything. Maximum reply tokens, maximum retries, timeouts. An unbounded loop around a paid, slow model is an outage waiting to happen.

For robots. Never let prose from a model drive an actuator. Ask for a structured command, validate it against the robot’s schema and limits (Lesson LM11-T4), and keep a safe default for the case where no valid command arrives in time.

Common misconceptions

Engineering takeaways

1. Base models continue text; instruction-tuned and chat models are trained to follow requests. Choose by task, and keep the base model for continuation, scoring and fine-tuning.
2. A prompt has a role, constraints, delimited inputs, examples and an output format; decoding settings (temperature 0, maximum length) belong to it.
3. Parse and validate every structured reply; retry with the error a bounded number of times; fall back safely. With per-attempt validity r the success within m tries is 1 − (1 − r)^m.
4. Version prompts and the model, keep a test set, and score changes with the methods of Module 9.
5. Prompt wording reduces failures but is not a control; enforce what must hold outside the model.

References and further reading

Module 7 of this track covers instruction tuning in detail; Module 9 the evaluation methods used here to compare prompts. The retry arithmetic assumes independent attempts and is an optimistic bound.

• Ouyang, L. et al. (2022). Training language models to follow instructions with human feedback. arXiv:2203.02155.
• Brown, T. et al. (2020). Language models are few-shot learners. arXiv:2005.14165.
• Wei, J. et al. (2022). Finetuned language models are zero-shot learners. arXiv:2109.01652.

Knowledge check

Three conceptual questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

What’s next

Next: embeddings and semantic similarity, the basis for retrieval.

Related content

Nothing below is required to finish this lesson.

• Before
  LM10-T6: Inference Efficiency and Architecture Trade-Offs

  Combine quantization, batching and architecture choices into a latency and memory budget.

• Lab
  LM11-L2: Validate Structured Outputs and Tool Calls

  Parse model output into a schema, reject invalid or unsafe calls and execute only validated actions.