Ratcheting Modular Intelligence

Original Corben Sorenson paper published as part of The ASI Stack source and lineage library.
Author

Corben Sorenson — original collaborator credits preserved in the manuscript

Published

Invalid Date

← Corben Papers and Architecture Sources

ImportantOriginal paper, not rewritten book prose

This page publishes Corben Sorenson’s original source manuscript so readers can inspect the ideas that preceded or informed the living book. The text may contain historical terminology, claims, confidence, citations, or implementation status that the book later narrows, revises, tests, or rejects. Publication here establishes provenance and access—not correctness, novelty, replication, or support-state promotion.

Publication and provenance

Field Record
Source ID rmi
Source class author paper
Library class architecture_whitepaper
Manuscript date Date not normalized
Inventory updated Not separately recorded
Exact published-source SHA-256 1ab708ee4f0df25e66237b641e6da21bb847377af1d08ceeab7a2ca9ac19a35f
Exact published-source bytes 49,750
Exact source text Download/view the tracked Markdown source
Book’s source note Read the bounded mining note
Authorship and collaborator credits Preserved from the exact original manuscript; this library wrapper does not replace or simplify them.
Rights No new license grant. Corben Sorenson’s rights are reserved; collaborator, quotation, source-title, and third-party rights remain with their holders.

Current publication boundary. Archived author paper; its claims retain the status and limits stated in the paper and do not inherit the living book’s current evidence state.

HTML presentation note. The HTML page normalizes line endings and trailing whitespace, preserves explicit Markdown hard breaks, and demotes manuscript headings beneath the page title. The digest above applies to the linked exact source text, not to this presentation wrapper.

Where this paper enters the living book

The Efficient ASI Hypothesis, Capability Replacement and Rollback, Recursive Self-Improvement Boundaries, Open-Ended Improvement Engines, Embodied Agency, Real-Time Control, and Physical Safety, Procedural Memory and Cognitive Loop Closure, Routing Heads and Specialist Cores, Replaceable Cognitive Substrates: Beyond Transformer Monoculture, Readiness Gates, Residual Escrow, and Quarantine, Personal Compute Hives and Federated Edge Intelligence, Compact Generative Systems: Generate, Verify, Repair, and Residual Honesty, Fast Generation Architectures, Benchmark Ratchets and Anti-Goodhart Evidence, Adversarial Evaluation, Sandbagging, and Training-Time Deception, Policy Optimization and Learning from Feedback, Artifact Steward Agents and Living Project Governance, Integrated Reference Architecture, Open Research Agenda and Bibliography Plan

Original manuscript

Ratcheting Modular Intelligence A Unified Framework for Active Compression, Loop Closure, Benchmark Frontiers, and Routed Specialist Systems White Paper Public Release v1.0 — May 2026 Author: Corben Sorenson Status: Conceptual Framework + AI Systems Architecture Proposal + Development Methodology ________________

TL;DR / Executive Abstract Modern AI systems are powerful, but their development is fragmented. We scale models, benchmark them, attach tools, add memory, create agents, write safety policies, and occasionally change architectures. These pieces often work, but they are rarely unified into one coherent growth process. This paper proposes Ratcheting Modular Intelligence, or RMI: a framework for AI systems that improve by turning pressure into structure. The core thesis is: AI systems should grow by compressing experience into verified modular structure, then using benchmark pressure to ratchet that structure toward harder capabilities. RMI combines five ideas into one standalone framework: Idea Role Compact generative structure Capabilities become durable when represented as small, inspectable structures that can generate, predict, control, or govern larger behaviors. Active compression Experience is compressed into memory, tools, policies, benchmarks, residual maps, and architecture changes that make future behavior easier to generate and verify. Cognitive loop closure Repeated reasoning/action trajectories are compiled into verified, parameterized tools. Benchmark ratcheting Benchmarks apply pressure until mastery, then become regression tests while harder benchmarks define the next frontier. Octopus routing A lightweight head/router dynamically composes specialized, independently improving modules rather than forcing every capability into one monolithic blob. The combined system works like this: benchmark frontier ↓ head/router receives task ↓ specialist arms attempt subtasks ↓ successes and failures are logged ↓ repeated successful trajectories become tools ↓ failures become residual maps and residual escrow ↓ benchmarks diagnose whether the wall is data, training, inference, tooling, evaluation, or architecture ↓ arms and router improve independently ↓ mastered benchmarks become regression tests ↓ new harder benchmarks become the frontier ↓ repeat

A mature RMI system is not a single giant model trying to know everything. It is one coherent external agent composed of many bounded internal specialists. The metaphor is an octopus. The head/router coordinates, but the arms are locally capable. Octopus arms are biologically rich control structures: recent cephalopod work notes that octopus arms are muscular hydrostats with extremely high degrees of freedom and hundreds of chemotactile suckers, and that more neurons are distributed across the arms than in the brain. This makes the octopus a useful analogy for distributed intelligence: one organism, many capable limbs, local control, global coordination. (Nature) The informal metaphor is: Goblins in a trenchcoat. Externally, the user sees one agent. Internally, many specialized critters coordinate. This framework argues against three failure modes: 1. Monolithic scaling: assuming every capability must live in one increasingly large model. 2. Static benchmarking: treating fixed benchmarks as permanent definitions of intelligence. 3. Repeated improvisation: forcing agents to reason through the same workflow forever. Instead, RMI proposes: Reason when novel. Execute when closed. Reflex when safety-critical. Ratchet when measured. Split when bloated. Retire when stale. The system has three execution modes: Mode Purpose Interpreter mode Flexible reasoning for novel, ambiguous, creative, or underspecified tasks. Compiled-tool mode Verified tool execution for repeated, parameterized, well-understood tasks. Reflex/failsafe mode Immediate safety behavior when latency or risk makes reasoning too slow. It also has a benchmark curriculum policy. A model or arm does not need 100% on every ordinary benchmark before moving forward. A benchmark can have a high initial mastery threshold, such as 90%, that decays over stalled effort toward a floor, such as 70%, for non-safety-critical tasks. The unsolved remainder enters residual escrow, where it is tracked, periodically reattempted, and promoted back into active development if it recurs across later benchmarks. The key rule is: The frontier must move, but the floor must hold. RMI therefore maintains: * a head/router; * a set of specialist arms; * an arm registry; * a benchmark ledger; * a model ledger; * a tool registry; * a residual escrow ledger; * a public calibration track for apples-to-apples comparison; * a safety/reflex layer; * a routing memory; * an intervention ladder that decides whether the next improvement should come from data, training, inference, loop closure, benchmark repair, bridge benchmarks, or architecture change. The final claim is: The best AI systems will not merely become larger. They will become better organized. They will turn pressure into structure, repetition into tools, failures into diagnostics, and benchmarks into curriculum. That is Ratcheting Modular Intelligence. ________________

Abstract AI progress is commonly described through scaling, benchmarks, post-training, tool use, agent scaffolding, memory, and architecture design. These are usually treated as separate concerns. This paper proposes a unified framework: Ratcheting Modular Intelligence. Ratcheting Modular Intelligence is a framework for AI systems that improve by repeatedly confronting unsaturated benchmark frontiers, analyzing residual failures, converting repeated successful trajectories into verified parameterized tools, preserving mastered capabilities as regression tests, and routing work across specialized modules that can be independently improved, split, merged, retired, or dynamically loaded. The framework synthesizes five mechanisms. First, compact generative structure: a useful capability becomes durable when it is represented as a small, inspectable structure that can generate, predict, control, or govern a larger class of behaviors. Second, active compression: experience is compressed into memory, tools, procedures, residual maps, benchmarks, policies, and architectures that reduce future uncertainty and effort. Third, cognitive loop closure: repeated reasoning/action trajectories become verified tools rather than being re-inferred from scratch. Fourth, benchmark ratcheting: benchmarks act as temporary pressure surfaces. Once mastered, they become regression tests; unresolved cases enter residual escrow; harder benchmarks define the next frontier. Fifth, octopus routing: a lightweight learned head/router coordinates dynamically loaded specialist arms with local tools, memory, benchmarks, permissions, and verification contracts. The paper defines the formal model, execution modes, benchmark lifecycle, time-decayed mastery thresholds, residual escrow, public calibration, loop-closure pipeline, octopus router architecture, specialist arm lifecycle, cognitive substrate options, safety and runtime tiers, ledgers, metrics, failure modes, and implementation roadmap. RMI draws on existing research while proposing a system-level synthesis. Minimum Description Length frames learning as compression through model-plus-data description length. HELM emphasizes broad, multi-metric evaluation of language models. Toolformer shows that language models can learn when and how to call external tools. Voyager demonstrates an embodied LLM agent with an executable skill library. Mixture-of-Experts systems show the value of sparse expert activation inside neural networks. Liquid Time-constant Networks, KANs, HDC/VSA, and active inference provide candidate primitives for continuous state, inspectable transformations, compositional memory, and action selection. (Stanford CRFM) The contribution of this paper is not any one primitive. The contribution is the architecture of growth: Use benchmarks to expose missing capability, use loop closure to preserve repeated success, use modular routing to localize capability, and use verification to make progress durable. ________________

  1. Introduction 1.1 The fragmentation problem Modern AI systems are often built from pieces that do not fully cohere. A model is trained. A benchmark is run. A tool is added. A memory store is attached. A workflow is scripted. A safety policy is layered on top. A benchmark saturates. A new architecture is proposed. A new agent scaffold is built. Each piece may help, but the development process can become fragmented. The result is often an AI system that is powerful but poorly organized. It may gain new capabilities while losing old ones. It may overfit public benchmarks. It may reason through repeated tasks from scratch. It may accumulate tools without lifecycle management. It may use the wrong specialist for a task. It may fail to distinguish data problems from architecture problems. It may scale a monolith when it needed a module. It may automate a loop before discovering hidden parameters. It may treat safety as an afterthought. RMI proposes that these problems share one missing structure: AI systems need a ratcheting architecture of growth. ________________

1.2 The central pattern Across all the ideas developed in this paper, the same pattern appears: pressure ↓ attempt ↓ residual ↓ compression ↓ verification ↓ structure ↓ new pressure

A benchmark applies pressure. The system attempts the benchmark. Failures become residuals. Repeated successes become tools. Tools and memories become compact structures. Verified structures become the floor for the next level. The next benchmark becomes the frontier. The system grows by ratchet. ________________

1.3 Why benchmarks alone are not enough Benchmarks are necessary, but they decay. A benchmark can become too easy, too public, too contaminated, too narrow, too noisy, or too poorly aligned with the capability we care about. This is already visible in frontier AI. Humanity’s Last Exam was introduced partly because models had exceeded 90% accuracy on popular benchmarks such as MMLU, reducing their usefulness as frontier measures. (Live Science) OpenAI argued in 2026 that SWE-bench Verified had become increasingly contaminated and no longer measured frontier coding capabilities well, recommending SWE-bench Pro instead. (OpenAI) Benchmarks should therefore not be worshiped. They should be used as pressure surfaces. When they stop applying useful pressure, they should become regression tests or be retired. ________________

1.4 Why tool use alone is not enough Tool use is useful, but ordinary tool use assumes tools already exist. A more mature agent should notice when it repeatedly performs the same workflow and create a tool from that workflow. This is cognitive loop closure: repeated trajectories ↓ abstraction ↓ parameterization ↓ tool synthesis ↓ verification ↓ routing

Toolformer showed that language models can learn when and how to call external APIs, but RMI extends the question from tool use to tool formation: when should repeated agent behavior become a new verified tool? (Hugging Face) ________________

1.5 Why monolithic scaling alone is not enough Large models are powerful. This paper does not deny that. But forcing all capability into one model creates problems of memory, specialization, interpretability, permissioning, and upgradeability. The alternative is not to abandon general intelligence. The alternative is to organize it. RMI proposes an octopus router architecture: * one coherent external agent; * many internal specialists; * a lightweight head/router; * dynamically loaded arms; * local memory; * local tools; * local benchmarks; * local permissions; * local residuals; * head-level composition. The system scales by organization as well as parameter count. ________________

  1. Definition of Ratcheting Modular Intelligence A Ratcheting Modular Intelligence system is an AI system that improves by converting benchmark pressure, repeated behavior, and residual failures into verified modular capability. It has seven core objects: Object Meaning Head/router The coordinating system that interprets tasks, selects specialists, allocates budget, composes outputs, and enforces routing policy. Arms/specialists Domain-specific modules with local tools, memory, permissions, benchmarks, and verification contracts. Benchmark frontier The current unsaturated evaluation pressure surface. Regression suite Previously mastered benchmarks and cases used to preserve capability. Trajectory log Records of attempts, actions, tool calls, outputs, failures, and verification outcomes. Tool registry Verified parameterized tools compiled from repeated successful trajectories. Residual escrow Tracked unresolved failures, edge cases, benchmark defects, and recurring weaknesses. The system’s core loop is: frontier benchmark ↓ head/router decomposes task ↓ arms execute subtasks ↓ results are verified and composed ↓ successes and failures are logged ↓ repeated successes close into tools ↓ failures enter residual maps or escrow ↓ intervention ladder selects next improvement ↓ benchmarks graduate or remain frontier ↓ arms/router/architecture ratchet upward

The framework’s central law is: Improve the frontier without losing the floor. ________________

  1. Formal Model Let the system at development cycle (t) be: \[ S_t = (H_t, \mathcal{A}_t, \mathcal{R}_t, \mathcal{M}_t, T_t, B_t, G_t, E_t, V_t) \] where: Symbol Meaning (H_t) Head/router. (t) Set of arms/specialists. (t) Routing policy. (t) Memory system. (T_t) Tool registry. (B_t) Active benchmark frontier. (G_t) Regression suite. (E_t) Residual escrow and failure map. (V_t) Verification and safety layer. Given task (x), context (c), budget (q), and risk profile (r), the head selects arms: \[ A_x = \mathcal{R}_{H_t}(x, c, q, r) \] where: \[ A_x \subseteq \mathcal{A}_t \] Each selected arm (A_i) receives scoped context and permissions: \[ y_i = A_i(x_i, c_i, p_i) \] where (p_i) is the permission/resource envelope. The head composes: \[ y = H_{\text{compose}}(y_1, y_2, \dots, y_k) \] The verifier evaluates: \[ V_t(y, x, c, r) \rightarrow {\text{accept}, \text{revise}, \text{route more}, \text{fallback}, \text{refuse}} \] The ratchet advances when: \[ P(S_{t+1}, B_t) > P(S_t, B_t) + \epsilon \] while: \[ P(S_{t+1}, G_t) \geq P(S_t, G_t) - \delta \] In plain language: The new system must improve the frontier while preserving prior mastered capability. _____________

  2. The Five Pillars 4.1 Compact generative structure The first pillar is that capability becomes durable when compressed into a structure that can generate, predict, control, or govern behavior. A compact structure may be:

  • a model;
  • a tool;
  • a policy;
  • a benchmark;
  • a memory state;
  • a schema;
  • an arm;
  • a router rule;
  • an architecture;
  • a verifier;
  • a reflex controller. The generic form is: \[ \mathcal{C} = (S, R, M, \epsilon, V, G) \] where: Symbol Meaning
    1. Seed or compact core.
    2. Rule system or expansion process.
    3. Memory or state. () Residual/error.
    4. Verification.
    5. Generation or governance interface. This framing comes from the idea that the best structures do not merely describe. They do work. ________________

4.2 Active compression The second pillar is active compression. A model compresses training data into parameters. An agent compresses experience into memory. A tool compresses repeated action into procedure. A policy compresses many possible futures into action. A benchmark compresses a capability target into a test. A residual map compresses failures into a diagnostic signal. RMI treats intelligence as the active transformation of experience into reusable generative structure. ________________

4.3 Cognitive loop closure The third pillar is loop closure. A mature AI system should not reason through the same routine indefinitely. Repeated cognition should become procedural memory. Let successful trajectories be: \[ \mathcal{L} = {\tau_1, \tau_2, \dots, \tau_n} \] A loop closure engine compiles them into a tool: \[ {\tau_1, \tau_2, \dots, \tau_n} \rightarrow T_\phi(p) \] where (p) captures task-specific variation. The tool is accepted only if: \[ \operatorname{Verify}(T_\phi) \geq \theta \] The trajectory-to-tool move is what lets agents gain skill without hiding everything inside weights. ________________

4.4 Benchmark ratcheting The fourth pillar is benchmark ratcheting. Benchmarks are not permanent definitions of intelligence. They are temporary pressure surfaces. A benchmark begins as frontier. If mastered, it becomes regression. If noisy, it is repaired or retired. If too hard, bridge benchmarks are created. If public, it is used for calibration but not as the only truth. HELM’s evaluation philosophy is relevant here because it emphasizes broad coverage, explicit recognition of incompleteness, and multi-metric measurement rather than evaluating only one accuracy score. (Stanford CRFM) ________________

4.5 Octopus routing The fifth pillar is modular routing. Instead of forcing every capability into one blob, RMI uses: * a head/router; * specialist arms; * dynamic loading; * memory routing; * permission routing; * local benchmarks; * local residual escrow; * head-level composition. This borrows sparse activation intuition from Mixture-of-Experts systems while moving the idea from neural layers to the system architecture. Switch Transformers, for example, use sparse expert routing to scale model capacity without activating all parameters for every input. (Hugging Face) RMI generalizes this: Route tasks to governed specialist systems, not only tokens to expert layers. ________________

  1. Architecture Overview The RMI architecture contains nine major subsystems:
  2. Head/router
  3. Arm registry
  4. Specialist arms
  5. Memory router
  6. Permission router
  7. Tool registry
  8. Benchmark and regression ledger
  9. Residual escrow ledger
  10. Verification and safety layer A simplified flow: User / Environment ↓ Head Router ↓ Task decomposition + risk + budget ↓ Arm Registry / Memory Router / Permission Router ↓ Dynamically loaded specialist arms ↓ Arm outputs + confidence + residuals + provenance ↓ Head composition ↓ Verification / safety review ↓ Response or action ↓ Logs update tools, benchmarks, residuals, and routing policy
  1. The Head/Router 6.1 Role The head/router is the coordinating intelligence. It handles:
  • user interaction;
  • task interpretation;
  • decomposition;
  • risk assessment;
  • arm selection;
  • dynamic loading;
  • budget allocation;
  • memory routing;
  • permission routing;
  • output composition;
  • safety escalation;
  • detecting when no arm is sufficient;
  • recommending arm spawn/split/merge/retirement. The head does not need to be the best specialist in every domain. It needs to know how to allocate specialists. ________________

6.2 Router benchmarks The router needs its own evaluations. Metric Meaning Selection accuracy Did it choose the right arm? Abstention quality Did it avoid routing when no arm fit? Cost efficiency Did it avoid unnecessary arms? Risk routing Did high-risk tasks trigger safety arms? Composition quality Did it faithfully synthesize arm outputs? Conflict resolution Did it handle disagreement well? Latency compliance Did it respect time budgets? Arm discovery Did it detect when a new arm was needed? The router should be treated as a first-class model with its own residuals and regression tests. ________________

6.3 Router non-monolith rule The head must not absorb all domain reasoning. If the head becomes responsible for everything, the system collapses back into monolithic design. The head should specialize in: * routing; * composition; * verification orchestration; * global coherence. Domain reasoning belongs in arms. ________________

  1. Specialist Arms 7.1 Definition An arm is a bounded specialist subsystem. An arm may be:
  • a small model;
  • a domain-tuned model;
  • a workflow engine;
  • a retrieval system;
  • a code executor;
  • a verifier;
  • a symbolic reasoner;
  • a safety monitor;
  • a physical controller;
  • a tool bundle;
  • a hybrid subsystem. The key property is not implementation type. The key property is bounded, evaluated specialization. ________________

7.2 Arm anatomy Each arm should have: Component Purpose Capability scope What the arm handles. Input/output schema How the head communicates with it. Local tools APIs, commands, functions, or environments. Local memory Domain-specific history and state. Local benchmark frontier Current pressure surface for that arm. Regression suite Prior capabilities to preserve. Residual escrow Local unresolved failures. Permission boundary What the arm can access or change. Runtime tier Where and how it executes. Reliability metrics Success, failure, cost, latency. Lifecycle status Active, probationary, split candidate, stale, retired. An arm is not a prompt. It is a governed subsystem. ________________

7.3 Arm examples Arm Scope Coding Arm Repository analysis, code edits, tests. Rust Arm Rust compiler errors, Cargo, ownership, lifetimes. Research Arm Literature mapping, citation support, related work. Math Arm Derivations, formal calculations, proof checking. Data Arm Tables, spreadsheets, normalization, analysis. Writing Arm Structure, tone, public-release polish. Skeptic Arm Overclaim detection and adversarial critique. Safety Arm Risk classification, vetoes, permission review. Memory Arm Retrieval, project history, user preferences. Vision Arm Image/video interpretation. Operations Arm Deployments, incidents, monitoring. Reflex Arm Immediate safety response in hard-latency contexts. ________________

  1. Dynamic Loading and Domain Quarantine 8.1 Dynamic loading The architecture keeps the head/router loaded and loads arms on demand. head stays resident arms load when needed

Benefits: * lower active memory footprint; * rare skills can remain cold; * domain-specific systems can run on specialized hardware; * sensitive arms can remain isolated; * one arm can update without retraining the entire system; * fault containment improves. ________________

8.2 Domain quarantine Different arms should have different memory and permissions. Examples: * Coding Arm can read repositories but cannot send emails. * Finance Arm can calculate but cannot approve transactions without permission. * Medical Arm can provide general information but cannot access unrelated personal data. * Deployment Arm can dry-run but requires approval for production mutation. * Safety Arm can veto high-risk actions. Each arm receives a permission envelope: \[ p_i = (\text{memory}, \text{tools}, \text{runtime}, \text{side effects}, \text{budget}, \text{risk}) \] The arm operates only inside that envelope. ________________

8.3 Memory routing Memory should be routed like compute. Memory type Purpose Global memory User preferences, long-term goals, persistent identity. Arm-local memory Domain-specific cases, tools, failures, and heuristics. Shared task memory Temporary workspace for multi-arm collaboration. Routing memory Which arms worked for which tasks. Safety memory Incidents, approvals, vetoes, risk history. Residual memory Unresolved failures and edge cases. The principle: Do not expose all memory to all arms. Route memory by task, permission, and need. ________________

  1. Cognitive Loop Closure 9.1 Why loop closure matters A system that repeats the same reasoning forever is not learning efficiently. If an arm or the head repeatedly performs the same workflow, it should ask: Can this become a verified tool? Examples:
  • repository test runner;
  • citation audit;
  • public whitepaper preparation;
  • spreadsheet normalization;
  • deployment hold;
  • invoice processing;
  • prompt-to-eval-suite generator;
  • data-cleaning procedure. ________________

9.2 Loop closure pipeline The loop closure pipeline is: 1. Trajectory logging 2. Loop detection 3. Abstraction 4. Active parameter discovery 5. Tool synthesis 6. Verification 7. Registration 8. Routing 9. Runtime monitoring 10. Revision or retirement This pipeline prevents naïve automation by requiring verification and lifecycle management. ________________

9.3 Active parameter discovery Passive abstraction is not enough. If every observed invoice is in USD, currency may look invariant. If every observed repository uses npm, package manager may look invariant. If every observed paper includes references, reference presence may look invariant. The system must actively probe for hidden parameters. Methods include: * historical variance analysis; * counterfactual replay; * synthetic case generation; * adversarial edge-case probing; * environment interrogation; * human or supervisory questioning. Variables should be classified as: State Meaning Invariant Should not change across valid uses. Parameter Expected to vary and should be exposed. Precondition Must hold before execution. Unknown assumption Suspected dependency requiring more evidence. This prevents tools from becoming brittle scripts. ________________

9.4 Tool acceptance rule A tool should enter the registry only when expected value exceeds lifecycle cost: \[ F \cdot \Delta C \cdot Q \cdot A C_T + M_T + R_T + V_T + D_T \] where: Symbol Meaning (F) Expected recurrence frequency. (C) Expected cost reduction per use. (Q) Expected quality or reliability improvement. (A) Automation appropriateness. (C_T) Creation cost. (M_T) Maintenance cost. (R_T) Risk cost. (V_T) Verification cost. (D_T) Drift/depreciation cost. This prevents tool bloat. ________________

  1. Benchmark Ratcheting 10.1 Benchmark lifecycle Benchmarks move through statuses: Status Meaning Frontier Current pressure benchmark. Diagnostic Explains a failure mode. Graduated Mastered enough to advance. Regression Preserves prior capability. Public calibration Enables apples-to-apples comparison. Live Refreshes over time. Retired Too stale, noisy, contaminated, or uninformative. A benchmark is useful while it teaches. ________________

10.2 Mastery thresholds A model or arm does not need 100% on every ordinary benchmark. Each benchmark (b) has an initial mastery threshold: \[ \gamma_{0,b} \] For many ordinary capability benchmarks, a useful default may be: \[ \gamma_{0,b} = 0.90 \] or 90%. Graduation requires: \[ P(S_t,b) \geq \gamma_b \] but also subgroup and critical-failure checks. ________________

10.3 Time-decayed thresholds If progress stalls, the threshold may decay toward a floor: \[ \gamma_b(k) \max \left( \gamma_{\min,b}, \gamma_{0,b} - \eta_b \cdot \max(0, k - p_b) \right) \] where: Symbol Meaning (b(k)) Current threshold. ({0,b}) Initial threshold. ({,b}) Minimum floor. (k) Development cycles spent on benchmark. (p_b) Patience window before decay. (b) Decay rate. A cautious policy decays only when improvement stalls: \[ \Delta_b(k) < \epsilon_b \] The purpose is not to lower standards randomly. It is to prevent one benchmark tail from freezing the frontier forever. ______________

10.4 Critical-failure veto Time-decayed thresholds do not apply to critical safety failures. Graduation requires: \[ F_{\text{critical}}(S_t,b)=0 \] or a domain-specific near-zero bound. Safety-critical failures include: * irreversible physical harm; * security compromise; * unsafe deployment; * financial harm; * medical harm; * legal violation; * catastrophic data loss. ________________

10.5 Residual escrow When a benchmark graduates, unsolved cases enter residual escrow: \[ E_b = {x \in b \mid S_t(x) \neq y} \] Escrow items are: * tracked; * clustered; * periodically reattempted; * promoted if recurring; * retired if benchmark-flawed; * added to regression if solved consistently. The rule: Advance at mastery. Preserve the tail. Promote recurring residuals. ________________

  1. Public Calibration Track Internal benchmarks drive progress. Public benchmarks enable comparison. A mature system should use both. Track Purpose Internal frontier Drive capability growth. Diagnostic Explain failures. Private holdout Test generalization. Live benchmark Track real-world performance. Regression suite Preserve mastered capability. Residual escrow Track unresolved failures. Public calibration Compare to public reports. Public calibration answers: How do we compare to the field? Internal frontier benchmarks answer: What should we improve next? Both are necessary. OpenAI’s SWE-bench Verified analysis illustrates why public benchmarks need lifecycle management: a benchmark can be useful, then become less diagnostic as contamination and flawed residual tests dominate. (OpenAI) ________________

  2. Intervention Ladder When performance stalls, RMI escalates carefully. Level 1 — Benchmark audit Ask:

  • Are labels correct?
  • Are tests fair?
  • Is the benchmark contaminated?
  • Are tasks solvable?
  • Does the metric match the capability?
  • Does improvement transfer? Level 2 — Data improvement Try:
  • targeted examples;
  • label cleaning;
  • curriculum;
  • synthetic cases;
  • demonstrations;
  • better coverage. Chinchilla-style compute-optimal training showed that some apparent model limits are actually data/training-allocation problems, with model size and training tokens needing to scale together under fixed compute. Level 3 — Training improvement Try:
  • loss changes;
  • optimizer tuning;
  • curriculum;
  • post-training;
  • preference learning;
  • reinforcement learning;
  • distillation. Level 4 — Inference improvement Try:
  • retrieval;
  • memory;
  • search;
  • planning;
  • test-time compute;
  • verifiers;
  • tool use;
  • decomposition. Level 5 — Loop closure If successful workflows repeat, compile them into tools. Level 6 — Bridge benchmarks If the next benchmark is too hard, insert an intermediate benchmark. Level 7 — Architecture change If data, training, inference, tools, benchmark repair, and bridge benchmarks fail, change architecture. Architecture should be a hypothesis: This residual exists because the system lacks mechanism (X). Adding (X) should improve benchmark class (B) while preserving regression suite (G). ________________

  1. Reference Cognitive Substrate RMI does not require one specific model architecture. However, a strong cognitive substrate for RMI may combine five primitives: Primitive Role Liquid continuous state Streaming temporal memory. Reservoir expansion Cheap nonlinear temporal basis. KAN-style transformations Inspectable nonlinear compression and readout. HDC/VSA memory Explicit compositional binding and symbolic memory. Active-inference-style action selection Choosing actions, queries, or tools that reduce expected uncertainty. Liquid Time-constant Networks provide a basis for continuous-time recurrent dynamics; KANs replace fixed node activations with learnable edge functions; Torchhd describes HDC/VSA as computing with high-dimensional distributed representations; active inference frames perception, planning, action, decision-making, and learning under a generative-model objective. (Hugging Face) In this paper, that substrate is optional. The central contribution is not any one module. The central contribution is the ratcheting architecture around it. ________________

  2. Execution Modes and Safety 14.1 Interpreter mode Use flexible reasoning when the task is:

  • novel;
  • ambiguous;
  • creative;
  • underspecified;
  • high-level;
  • outside known tool preconditions. 14.2 Compiled-tool mode Use verified tools when the task is:
  • repeated;
  • parameterized;
  • well-scoped;
  • validated by prior tests;
  • low or acceptable risk. 14.3 Reflex/failsafe mode Use immediate safety behavior when:
  • latency is hard;
  • physical safety is at stake;
  • deployment risk is high;
  • financial or security containment is needed;
  • a tool fails during execution;
  • a system approaches a safety boundary. Runtime verification research studies monitors that evaluate execution traces against formal specifications, and WebAssembly’s security model provides sandboxing and isolation goals for executable modules. These are relevant because RMI tools and arms need runtime boundaries, not merely promises. (Springer) ________________

  1. Execution and Runtime Tiers Tier Environment Appropriate for E0 Text template Low-risk drafting. E1 Structured workflow Human-reviewed procedures. E2 Typed deterministic function Data transformations, parsing, formatting. E3 Sandboxed runtime Generated or untrusted code. E4 Memory-safe systems runtime Higher-assurance digital tools. E5 Real-time reflex runtime Safety-critical embodied systems. Generated tools should run in the least powerful environment sufficient for the task. Rust is relevant to E4-style systems because its ownership model governs memory management with compiler-checked rules; WebAssembly is relevant to E3-style sandboxing because each module executes in a separated sandboxed environment with explicit security constraints. (WebAssembly) ________________

  2. High-Bandwidth Embodied Logging Embodied systems create high-bandwidth streams:

  • 60fps camera feeds;
  • lidar;
  • IMU;
  • motor commands;
  • force sensors;
  • GPS;
  • battery telemetry;
  • localization estimates;
  • controller states. A ratcheting embodied system should not feed raw streams directly into loop detection. It needs hierarchical logging: Log type Purpose Raw telemetry Replay, debugging, safety analysis. Event log Obstacle detected, gate passed, slip detected, reflex triggered. Semantic trace Objects, landmarks, task state, environment labels. Skill trace Which controller/tool/arm was active. Residual log Surprises, failures, monitor violations, recovery events. The memory budget should satisfy: \[ B_{\text{raw}} + B_{\text{events}} + B_{\text{features}} + B_{\text{residuals}} \leq B_{\text{budget}} \] The goal: Log enough structure to discover loops, enough detail to debug failures, and enough safety evidence to audit reflex behavior — without storing the entire world. ________________
  1. Arm Lifecycle Specialist arms should evolve. 17.1 Add arms Spawn a new arm when a recurring domain deserves its own specialist. Signals:
  • repeated routing failures;
  • recurring residual cluster;
  • high cost from generalist handling;
  • new domain demand;
  • repeated loop closures in the same domain;
  • need for distinct permissions or runtime. 17.2 Split arms Split an arm when it becomes bloated. Signals:
  • rising latency;
  • large internal tool count;
  • broad unrelated scope;
  • separate residual clusters;
  • declining reliability;
  • router confusion;
  • distinct risk domains. 17.3 Merge arms Merge arms when specialization adds little value. Signals:
  • overlapping tools;
  • same benchmark frontier;
  • same memory;
  • low usage;
  • redundant outputs;
  • maintenance cost exceeds value. 17.4 Retire arms Retire an arm when it is:
  • stale;
  • unused;
  • unsafe;
  • superseded;
  • failing regression;
  • too expensive;
  • no longer aligned with user needs. A healthy modular system adds, splits, merges, and retires. It does not only grow. ________________

  1. Ledgers and Registries 18.1 Benchmark ledger Field Meaning Benchmark name Identifier. Capability measured What it claims to test. Status Frontier, diagnostic, graduated, regression, live, public calibration, retired. Initial threshold Default mastery target. Current threshold Time-decayed target. Floor threshold Minimum threshold before diagnosis/bridge. Subgroup floors Minimum category requirements. Critical-failure rules Failures that veto graduation. Contamination risk Low, medium, high. Transfer evidence Whether progress generalizes. Escrow policy How failures are tracked. Retirement criteria When to stop using it. 18.2 Model/system ledger Field Meaning System version Identifier. Head version Router version. Arm set Active specialists. Architecture Model/substrate design. Data Training/post-training data. Inference procedure Tools, memory, retrieval, search, planning. Benchmark scores Full portfolio. Residual map Failure categories. Tool registry version Procedural memory state. Regression status Prior capabilities preserved? Cost profile Training, inference, latency, memory. Safety profile Risk evaluation. Next wall Current suspected bottleneck. 18.3 Tool registry Field Meaning Tool name Skill identifier. Source trajectories Prior successful workflows. Parameters Variable inputs. Preconditions When it may run. Postconditions What must hold after execution. Verification grade Confidence/testing level. Runtime tier Execution environment. Risk tier Consequence of failure. Usage metrics Frequency, success, savings. Failure modes Known residuals. Retirement criteria When to disable or revise. 18.4 Residual escrow ledger Field Meaning Residual ID Failure identifier. Source benchmark/tool/arm Where it came from. Failure type Data, training, inference, tool, benchmark, architecture, safety. Cluster Related failures. Severity Low, medium, high, critical. Reattempt schedule When to retry. Recurrence count How often it reappears. Promotion status Escrow, active diagnostic, regression, retired. ________________

  2. Evaluation Metrics 19.1 System metrics

  • end-to-end success;
  • cost per task;
  • latency;
  • active memory footprint;
  • arms loaded per task;
  • public calibration score;
  • regression preservation;
  • safety incident rate;
  • user usefulness;
  • residual trend. 19.2 Router metrics
  • correct arm selection;
  • abstention quality;
  • risk routing accuracy;
  • composition fidelity;
  • conflict resolution quality;
  • unnecessary routing rate;
  • latency compliance. 19.3 Arm metrics
  • local benchmark performance;
  • regression preservation;
  • residual escrow trend;
  • tool success rate;
  • memory footprint;
  • cost;
  • latency;
  • bloat index. 19.4 Tool metrics
  • recurrence frequency;
  • cost savings;
  • verification grade;
  • failure rate;
  • stale-tool rate;
  • tool overlap;
  • retirement rate. 19.5 Benchmark metrics
  • saturation level;
  • residual value;
  • transfer strength;
  • contamination risk;
  • public calibration relevance;
  • subgroup performance;
  • critical failure count. ________________
  1. Failure Modes 20.1 Benchmark gaming The system improves scores without improving real capability. Mitigations:
  • private holdouts;
  • live benchmarks;
  • benchmark mutation;
  • transfer checks;
  • capability narratives. 20.2 Tail obsession The system gets stuck chasing the last 10% of one benchmark forever. Mitigations:
  • time-decayed thresholds;
  • residual escrow;
  • frontier momentum rule;
  • bridge benchmarks. 20.3 Tail erasure The system advances and forgets unresolved failures. Mitigations:
  • residual escrow ledger;
  • recurring reattempts;
  • recurrence promotion rule. 20.4 Tool bloat The system creates too many tools. Mitigations:
  • tool acceptance rule;
  • usage metrics;
  • consolidation;
  • retirement. 20.5 Arm bloat A specialist becomes a hidden monolith. Mitigations:
  • split thresholds;
  • local benchmarks;
  • subdomain clustering;
  • bloat index. 20.6 Bad routing The head chooses the wrong specialist. Mitigations:
  • router benchmarks;
  • arm confidence;
  • verifier arms;
  • fallback routing. 20.7 Under-quarantine Arms access too much. Mitigations:
  • permission envelopes;
  • memory routing;
  • sandboxing;
  • audit logs. 20.8 Over-quarantine Arms cannot access enough to solve the task. Mitigations:
  • controlled access grants;
  • head-mediated retrieval;
  • escalation policies. 20.9 Reflex gap The system assumes it can fall back to reasoning when immediate action is required. Mitigations:
  • reflex/failsafe mode;
  • safety monitors;
  • emergency stop/hold/land/isolate policies;
  • hard runtime constraints. 20.10 Architecture churn The team changes architecture before diagnosing the wall. Mitigations:
  • intervention ladder;
  • residual map;
  • benchmark audit;
  • ablations. ________________
  1. Implementation Roadmap Phase 1 — Ledgers Build:
  • benchmark ledger;
  • model/system ledger;
  • residual escrow ledger;
  • basic tool registry. Goal: Know what exists, what fails, and what is preserved. Phase 2 — Basic router and arm registry Create:
  • head/router;
  • small arm set;
  • arm cards;
  • input/output schemas;
  • permission envelopes. Goal: Route tasks to bounded specialists. Phase 3 — Loop closure Add:
  • trajectory logger;
  • loop detector;
  • abstraction engine;
  • active parameter discovery;
  • tool synthesis;
  • verifier;
  • tool registration. Goal: Convert repeated success into procedural memory. Phase 4 — Benchmark ratchet Add:
  • mastery thresholds;
  • time-decay policy;
  • subgroup floors;
  • critical-failure vetoes;
  • residual escrow;
  • regression promotion. Goal: Move the frontier while preserving the floor. Phase 5 — Public calibration Run public benchmarks periodically with standardized settings. Report:
  • score;
  • cost;
  • latency;
  • tool use;
  • inference setup;
  • safety notes. Goal: Compare externally without overfitting public benchmarks. Phase 6 — Arm ratchets Give each arm:
  • local benchmark frontier;
  • local regression suite;
  • local residual escrow;
  • local improvement process. Goal: Improve specialists independently. Phase 7 — Safety/runtime tiers Add:
  • execution tiers;
  • risk tiers;
  • sandboxing;
  • runtime verification;
  • reflex/failsafe layer;
  • human approval gates. Goal: Prevent modular capability from becoming unsafe autonomy. Phase 8 — Arm lifecycle management Add:
  • spawn rules;
  • split rules;
  • merge rules;
  • retirement policy;
  • bloat metrics. Goal: Keep the modular ecosystem healthy. ________________

  1. Claims and Non-Claims 22.1 Claims This paper claims:

  2. AI systems should grow through a ratchet of benchmark pressure, residual analysis, loop closure, modular routing, verification, and frontier expansion.

  3. Benchmarks should be treated as temporary pressure surfaces, not permanent definitions of intelligence.

  4. Ordinary benchmarks need mastery thresholds and residual escrow, not automatic 100% requirements.

  5. Repeated successful trajectories should become verified tools when valuable, parameterizable, and safe.

  6. Capability can scale through modular organization, not only through monolithic parameter growth.

  7. A head/router plus specialist arms enables dynamic loading, domain quarantine, independent ratcheting, and better diagnostics.

  8. Architecture changes should be motivated by residuals after simpler interventions fail.

  9. Public benchmarks are necessary for calibration but insufficient as the sole internal development target.

  10. Mature AI systems need three execution modes: interpreter, compiled-tool, and reflex/failsafe.

  11. Capabilities become durable when they are measured, proceduralized, verified, registered, routed, and protected against regression. 22.2 Non-claims This paper does not claim:

  12. Monolithic models are obsolete.

  13. Benchmarks perfectly measure intelligence.

  14. Higher benchmark scores always imply real-world improvement.

  15. Every repeated action should become a tool.

  16. Tools should replace reasoning.

  17. Fine-tuning or scaling are obsolete.

  18. Verification is absolute in open worlds.

  19. Time-decayed thresholds are acceptable for safety-critical failures.

  20. Routing is easy.

  21. More arms always improve the system.

  22. Human judgment is unnecessary.

  23. One architecture fits every deployment. This framework is a development methodology and systems architecture, not a complete theory of intelligence. ________________

  24. Conclusion Ratcheting Modular Intelligence proposes a unified way to build AI systems that grow without becoming ungovernable blobs. The system begins with pressure. A benchmark exposes what the system cannot yet do. The head routes tasks to arms. Arms attempt subtasks. Repeated successes become tools. Failures become residuals. Residuals diagnose the wall. Benchmarks graduate into regression. Unsolved cases enter escrow. Specialists improve independently. Bloated specialists split. Stale specialists retire. The public calibration track keeps the system comparable. The safety layer keeps the system bounded. The frontier moves. The floor holds. pressure ↓ attempt ↓ residual ↓ procedure ↓ verification ↓ module ↓ regression ↓ frontier

The best AI systems will not merely become larger. They will become better organized. They will know when to reason, when to execute, and when to reflex. They will know when a benchmark has taught enough. They will preserve the failures that still matter. They will turn repeated behavior into tools. They will turn tools into procedural memory. They will turn specialists into modular capability. They will turn residuals into architecture signals. They will turn benchmarks into curriculum. The ratchet turns when yesterday’s frontier becomes today’s floor. That is Ratcheting Modular Intelligence. ________________

Appendix A — One-Paragraph Public Summary Ratcheting Modular Intelligence is a framework for AI systems that improve by turning benchmark pressure, repeated behavior, and residual failures into verified modular capability. A lightweight head/router coordinates specialist arms, each with local tools, memory, benchmarks, permissions, and residuals. Repeated successful trajectories become verified tools. Mastered benchmarks become regression tests. Unsolved cases enter residual escrow. Public benchmarks provide calibration, while private/live benchmarks drive internal progress. The system improves through an intervention ladder: benchmark audit, data, training, inference, loop closure, bridge benchmarks, and architecture change. The result is one coherent agent made of many bounded specialists that ratchet upward over time. ________________

Appendix B — Compact Manifesto Do not worship benchmarks. Use them. Let them apply pressure. When they stop teaching, promote them to regression. Do not demand perfection from every benchmark. Graduate at mastery. Preserve the tail. If the tail repeats, promote it. If behavior repeats, compile it. If the tool works, verify it. If the tool fails, expose the residual. If an arm bloats, split it. If an arm overlaps, merge it. If an arm grows stale, retire it. If the task is novel, reason. If the loop is closed, execute. If safety is at stake and time is short, reflex. The user sees one agent. Inside, the specialists get to work. The frontier must move. The floor must hold. The ratchet must turn. ________________

Selected References 1. Stanford CRFM, Holistic Evaluation of Language Models / HELM. (Stanford CRFM) 2. Hoffmann et al., Training Compute-Optimal Large Language Models. 3. Center for AI Safety, Scale AI, and collaborators, Humanity’s Last Exam. (Live Science) 4. OpenAI, Why SWE-bench Verified no longer measures frontier coding capabilities. (OpenAI) 5. METR, Task-Completion Time Horizons of Frontier AI Models. (Metr) 6. Shazeer et al., Sparsely-Gated Mixture-of-Experts. (Hugging Face) 7. Fedus, Zoph, and Shazeer, Switch Transformers. (Hugging Face) 8. Schick et al., Toolformer. (Hugging Face) 9. Wang et al., Voyager. (Hugging Face) 10. Hasani et al., Liquid Time-constant Networks. (Hugging Face) 11. Liu et al., KAN: Kolmogorov-Arnold Networks. (Hugging Face) 12. Heddes et al., Torchhd: HDC/VSA Library. (Journal of Machine Learning Research) 13. Da Costa et al., Active inference on discrete state-spaces. (ScienceDirect) 14. Olson, Schulz, and Ragsdale, Neuronal segmentation in cephalopod arms. (Nature) 15. WebAssembly Project, Security model and sandboxing documentation. (WebAssembly) 16. Sánchez et al., Runtime verification from advanced application domains. (Springer)