39 Recorded Evaluations
40 Recorded Evaluations
Proof by Difference becomes credible only when weak and repaired prompts are run against the same fixture and scored with the same rubric. This section preserves those runs, including non-wins when they occur.
Generated at: 2026-07-02T12:44:28.207381+00:00
40.1 Summary
| Case | Surfaces | Weak Outcome | Repaired Outcome | Outcome Delta | Reviewability Delta |
|---|---|---|---|---|---|
| Refactor Without Breaking Behavior | claude-code-safe, codex-cli-default | 2.0 | 2.6 | repaired prompts satisfied 0.6 more outcome checks on average | repaired prompts scored 3.4 reviewability points higher on average |
| Incident Diagnosis Without Fake Certainty | claude-code-safe, codex-cli-default | 2.6 | 3.0 | repaired prompts satisfied 0.4 more outcome checks on average | repaired prompts scored 3.0 reviewability points higher on average |
| API Design Without Hidden Compatibility Traps | claude-code-safe, codex-cli-default | 3.0 | 3.0 | weak and repaired prompts tied on outcome checks | repaired prompts scored 2.9 reviewability points higher on average |
| Online Migration Without Data Loss | claude-code-safe, codex-cli-default | 2.9 | 3.0 | repaired prompts satisfied 0.1 more outcome checks on average | repaired prompts scored 0.6 reviewability points higher on average |
| Test Generation Without Overfitting To Implementation | claude-code-safe, codex-cli-default | 2.6 | 3.0 | repaired prompts satisfied 0.4 more outcome checks on average | repaired prompts scored 2.4 reviewability points higher on average |
| Performance Tuning Without Micro-Optimization Drift | claude-code-safe, codex-cli-default | 2.5 | 3.0 | repaired prompts satisfied 0.5 more outcome checks on average | repaired prompts scored 3.8 reviewability points higher on average |
40.2 Rubric
Outcome scores count case-specific obligations such as passing fixture tests, naming planted causes, preserving invariants, avoiding dirty-data traps, and including rollback boundaries.
Reviewability scores use 0-2 per criterion: artifact boundary, invariants, output contract, verification, failure behavior, and assumption control. The reviewability score is secondary and review-oriented. It partly rewards prompt echo because repaired spells contain words such as invariant, verify, rollback, and assumption. Read it as an inspectability signal, not as direct work quality. Transcripts and outcome checks remain the primary evidence.