66  Proof by Difference

67 Proof by Difference

Proof by Difference is the grimoire’s evidence discipline. It compares a weak request and a repaired spell against the same task. The claim is not that the repaired spell guarantees a perfect answer. The claim is that it makes success obligations inspectable: artifact boundary, invariant, output contract, verification, and failure behavior.

Use each case as a replayable prompt-design test. If a future model or workflow makes the weak request perform as well as the repaired spell, record that. If the repaired spell fails, inspect which limb was underspecified.

Replay templates, recorded runs, and the six-case evaluation matrix live in Recorded Evaluations and in the repository’s evaluation examples.

Spell	Case	Weak Request	Expected Delta	Observed Results
Spell of Safe Refactoring	Refactor Without Breaking Behavior	Clean up this Python module and make it nicer.	The repaired spell narrows blast radius, makes invariants reviewable, and forces proof through tests instead of relying on claims of cleanliness.	Recorded runs
Spell of Bug Diagnosis from Logs	Incident Diagnosis Without Fake Certainty	Why are requests timing out?	The repaired spell prevents counterfeit certainty while still producing actionable next checks.	Recorded runs
Spell of API Design	API Design Without Hidden Compatibility Traps	Design an API for billing.	The repaired spell forces the design to expose compatibility, authorization, idempotency, and migration surfaces up front.	Recorded runs
Spell of Migration Without Data Loss	Online Migration Without Data Loss	Migrate users.birthdate from string to date.	The repaired spell turns one risky command into a reversible campaign with validation and dirty-data handling.	Recorded runs
Spell of Test Generation	Test Generation Without Overfitting To Implementation	Write tests for this function.	The repaired spell makes tests protect behavior instead of freezing incidental implementation shape.	Recorded runs
Spell of Performance Tuning	Performance Tuning Without Micro-Optimization Drift	Make this faster.	The repaired spell forces measurement before optimization and keeps rollback criteria attached to speed claims.	Recorded runs