Prompt Engineering That Survives Model Upgrades
Prompts written as incantations break every time a model updates. Prompts written as specifications — with structure, examples, and evals — carry across model versions and vendors with minimal rework.
Last reviewed Jul 7, 2026
1. Specify, don't incant
Do: State the role, the task, hard constraints, the exact output format, and include 2–3 representative input→output examples. Say what to do in edge cases (empty input, ambiguity, missing data).
Don't: Stack persuasion tokens — 'you are the world's best', 'this is very important', 'take a deep breath' — and tune by vibes.
Why: Instructional specificity transfers across model generations; persuasion hacks are artifacts of a particular checkpoint and silently stop working on the next one.
2. Demand structured output at the API level
Do: Use the provider's structured-output / tool-calling mode with a JSON schema, and validate the result against that schema before using it.
Don't: Ask for JSON in prose and then regex the response out of a markdown code fence.
Why: Schema-enforced output eliminates an entire failure class (truncated JSON, chatty preambles), and validation failures become retryable events instead of downstream corruption.
3. Build a golden-set eval before you need it
Do: Collect 30–100 real inputs with expected outputs or grading criteria per critical prompt. Run the suite on every prompt edit and every model upgrade; block the change if scores regress.
Don't: Judge prompt changes by eyeballing three responses in a playground.
Why: Without evals, every model upgrade is a leap of faith and every prompt tweak can silently break a case you fixed last month. With them, upgrades become routine engineering.
4. Pin versions; upgrade on your schedule
Do: Reference dated model snapshots in production and treat a model upgrade like a dependency bump: branch, run evals, compare cost and latency, then promote.
Don't: Point production at a floating 'latest' alias and find out about behavior changes from your users.
Why: Vendors improve models in ways that shift tone, formatting, and refusal behavior. Pinning turns surprise regressions into planned migrations.
5. Isolate untrusted content
Do: Wrap user input and retrieved documents in clear delimiters, tell the model they are data (not instructions), and keep security-critical rules in the system prompt.
Don't: Concatenate user text directly into the instruction stream.
Why: This is both a correctness and a security practice — it reduces prompt-injection surface and stops user phrasing from hijacking output format.
6. Treat prompts as code
Do: Store prompts in the repo, review changes in PRs, and tag each production response with the prompt version that generated it.
Don't: Edit prompts live in a vendor dashboard with no history, review, or rollback.
Why: When output quality shifts, the first question is 'what changed?'. If prompts live outside version control, that question has no answer.
Why prompts rot
Every model generation invalidates a slice of the prompt-hack folklore: phrases that mattered on one checkpoint do nothing on the next, formatting quirks change, and refusal boundaries move. Teams that encoded behavior in incantations rewrite prompts under pressure after each upgrade; teams that encoded behavior in specifications plus evals mostly just re-run the suite and ship.
The discipline is boring on purpose: specification, examples, structured output, version pinning, golden-set evals, git. It is the same playbook software engineering already uses for every other dependency — the only new part is writing graders for outputs that are text instead of numbers.