Skip to main content

61  Fast Generation Architectures

61.1 Chapter status

Field	Value
Chapter ID	fast-generation-architectures
Part	Part III - Routing, Compression, Representation, and Substrates
Status	conceptual
Manuscript maturity	v0.3 claim-proof program
Last updated	2026-07-23
Primary source records	43 assigned records: eleven local architecture and residual sources; eleven decoding, serving, recurrent, state-space, and diffusion comparators; thirteen heterogeneous-memory comparators; and nine prefix, prompt-module, disaggregated-KV, non-prefix-fusion, provider-contract, and semantic-response-cache records
Claim label	Design rationale
Evidence level	argument
Source queue	primary: cgs, cognitive_loop_closure, benchmaxxing; supporting: the local generation, verification, resource, context, residual, and architecture sources; variants/external literature: acceleration substrates, heterogeneous inference memory, exact-prefix and prompt-module caches, cache-aware scheduling, disaggregated and non-prefix KV reuse, current provider contracts, and semantic response caching
Source loading state	source notes: cgs, cognitive_loop_closure, benchmaxxing, planforge, verification_bandwidth, tokenmana, talos, vcm_public, spinoza, rmi, ext_speculative_decoding_2022, ext_multi_token_prediction_2024, ext_medusa_2024, ext_eagle_2024, ext_lookahead_decoding_2024, ext_layerskip_2024, ext_pagedattention_vllm_2023, ext_mamba_2023, ext_llada_2025, ext_scaling_dllms_2026, ext_recurrent_transformer_2026, kernel_english_residual_compiler, ext_airllm_2023, ext_deepspeed_inference_2022, ext_flexgen_2023, ext_hf_accelerate_big_model_inference_2026, ext_llama_cpp_memory_mapping_2026, ext_llm_in_flash_2024, ext_powerinfer_2024, ext_vattention_2025, ext_infinigen_2024, ext_specache_2025, ext_specoffload_2025, ext_atsinfer_2026, ext_openai_prompt_caching_docs_2026, ext_anthropic_prompt_caching_docs_2026, ext_gemini_context_caching_docs_2026, ext_vllm_automatic_prefix_caching_2026, ext_sglang_radixattention_2024, ext_prompt_cache_2024, ext_mooncake_2025, ext_cacheblend_2025, ext_azure_llm_semantic_cache_2026, precision_contract; raw cache: cgs, cognitive_loop_closure, benchmaxxing, planforge, verification_bandwidth, tokenmana, talos, vcm_public, spinoza, rmi
Test state	AsiStackProofs.FastGeneration retains three genuine finite countermodels. AsiStackProofs.FastGenerationRefinement adds a seventeen-declaration, eight-stage request-to-closure lifecycle with sixty routes. python3 scripts/validate_fast_generation_refinement.py independently reaches every route, rejects 51/51 non-accepting mutations, recompiles Lean, replays the exact 2-valid/4-invalid baseline harness, revalidates the three-route/four-task accounting bundle and 1-valid/6-invalid Theseus import, and digest-binds every consumed fixture and result. One verified witness and one fallback witness reach closure with no support assignment or external-effect authority. Thirty-five projection, assumption-restatement, and copied-summary declarations are physically retired. This is finite authored policy/conformance evidence only: no model-speed, natural-workload usefulness, verifier-independence, deployed-fallback, serving, transfer, SOTA, or support claim.

61.2 Drafting guardrail

The missing stack layer is the generation substrate that determines how much planning, verification, tool use, reflection, and routing can fit inside a fixed latency and compute budget. No measured speedups are reported here, no third-party benchmark results are reproduced, and the repository does not claim to contain a fast decoding implementation.

The browser-GPT note that motivated this layer is treated as author-intent and literature-queue context. Selected external primary papers have now been source-noted for the taxonomy, but their reported speedups, model-quality claims, code artifacts, and hardware results remain source-reported. The acceleration taxonomy stays at argument support until reproduced tests, stronger external-literature promotion criteria, or accepted evidence transitions justify movement.

Coming after generate-verify-repair, fast generation is the time-domain version of the same discipline. A faster path is valuable only when accepted output, verifier cost, repair cost, fallback, memory pressure, and task success improve after the whole route is counted.

61.3 Human Reading Path

Concrete lens. The smallest fully resident model meeting the task contract is the strongest baseline, followed by unified-memory and ordinary offload paths; streaming must beat them after transfer, verification, recovery, and residual cost.

Compression is not the only way speed matters in generation pipelines. The generation substrate itself has to be part of the architecture: how much planning, verification, routing, tool use, and repair can fit inside a fixed latency and compute budget. A faster decoder can help the stack, but it can also make weak plans, stale context, or skipped checks fail faster.

Speed and intelligence need to stay separate. Faster generation can make a stack more usable, but only if accepted output, verification cost, fallback, memory pressure, and task success are counted. Raw tokens per second are not evidence that the system kept the right obligations alive. The decisive question is whether the whole governed route delivers correct work sooner. Generation architecture therefore belongs beside evidence and routing, not under a narrow serving benchmark alone.

Fast paths matter most when they preserve the slow obligations that make output usable.

Latency gains count only after acceptance and fallback remain intact. Speed becomes progress only when the verifier still wins. Verification protects useful speed by keeping acceleration tied to acceptance evidence.

61.4 Problem

Fast-generation work has to reduce end-to-end time to useful accepted work without treating raw token rate, synthetic proxy speed, aggregate serving throughput, or a cheaper unverified path as intelligence, quality, safety, or evidence.

Autoregressive language models make one high-value promise and one expensive tradeoff. They can condition each next token on the committed past, but the ordinary inference path is serial: produce a token, append it, run another step, and repeat. That is not merely an implementation detail. In a governed ASI stack, every second spent emitting text competes with planning, retrieval, verification, tool execution, artifact construction, benchmark replay, and human review.

Fast generation therefore belongs in the architecture. It is the layer that asks which parts of output must be generated left to right, which parts can be drafted in parallel, which drafts can be verified cheaply, which uncertain spans should be repaired, and which tasks require slower modes because the risk of accepting bad output is too high.

61.5 Why existing approaches are insufficient

Standard autoregressive decoding, isolated speculative or parallel methods, serving-only optimizations, and raw throughput leaderboards do not define a matched consumer/workload/model/hardware boundary. They also fail to attribute verifier, repair, fallback, queueing, warmup, cache, memory, energy, human, and governance costs or show that a faster route preserves quality and rights under load, failure, drift, and transfer.

The failure is conceptual before it is technical. A model can emit many tokens quickly and still produce less useful cognition per second if more tokens are rejected, repaired, rerun, or audited. A serving system can improve aggregate throughput while leaving single-request latency unchanged. A speculative or diffusion system can look fast when proposed tokens are counted, then lose the advantage when accepted tokens, verifier cost, memory pressure, retries, and task success are counted.

The accounting unit has to be stricter:

effective_verified_tokens_per_second =
    accepted_or_verified_correct_tokens / wall_clock_time

useful_solution_per_second =
    pass_rate_or_verified_success / wall_clock_time

Those metrics are not enough by themselves, but they point in the right direction. The system cares about useful, accepted, governed output under a declared task and risk model, not just surface text velocity.

61.6 Core Claim

[fast-generation-architectures.core, label: Design rationale, support: argument] Fast-generation admission is consumer-, workload-, model-, hardware-, serving-policy-, and time-window-specific: a controller may route an eligible request through a named accelerated path only after prospectively binding the context, quality, risk, budget, metric, verifier, fallback, rollback, and expiry contracts; separating attempted, proposed, accepted, verified, delivered, and useful output; fully attributing queueing, prefill, decode, verification, repair, retry, fallback, cache, memory, bandwidth, energy, human, and governance burdens; and showing a meaningful end-to-end improvement over matched quality-equivalent baselines without violating safety, authority, rights, or residual gates. Raw tokens per second, FLOP estimates, aggregate throughput, synthetic templates, or unverified speed lifts alone cannot qualify a route.

The claim remains at argument. It does not say that one acceleration family wins everywhere, and it does not let planning or routing self-qualify a mode. Fast Generation owns the request-specific admission and end-to-end speed receipt; Planning supplies the request, Routing supplies candidates, Verification supplies bounded verdicts, Benchmaxxing compares complete attempt universes, and Readiness or Claim Ledgers own support movement.

61.6.1 Publication placement and preserved technical ownership

This chapter is the stable technical-detail route beneath Resource Economics and Token Budgets. The parent chapter compares the complete cost of accelerated generation, deliberation, direct generation, verification, repair, fallback, and displaced work. This chapter continues to own generation-family mechanisms, cache and paging objects, proposal/acceptance accounting, workload-specific qualification, matched baselines, and fallback behavior. Publication nesting transfers no claim, source, proof, test, evidence, or authority to Resource Economics, and it does not change this chapter’s argument support ceiling or stable URL.

61.6.2 Claim-source mapping status

Appendix C carries exact bounded mappings for the original twenty-one assigned sources. Thirteen heterogeneous-memory records now add metadata-first drafting context and manuscript comparators without moving support. None supports a local decoder, serving system, paging implementation, verifier-preserving speed result, hardware transfer, or promoted generation mode.

Source	What it supports	Limit
cgs	Compact generative leverage with residual burden, verification cost, fallback, and governance interface visible.	No local decoding-speed, accepted-token, or useful-solution-per-second evidence.
cognitive_loop_closure	Repeated-generation shortcuts as procedural-memory pressure through logging, loop detection, parameterized tool synthesis, verification, registry, routing, monitoring, and retirement.	No local loop detector, tool synthesis run, fast-path cache, or procedural generation shortcut has been executed.
benchmaxxing	Benchmark lifecycle, wall diagnosis, ledgers, regression preservation, anti-Goodhart safeguards, transfer checks, and architecture-change criteria.	No benchmark harness, mutation, holdout, acceptance-rate test, or reproduced performance run exists here.
planforge	Planner-selected generation modes through decomposition, primitive schemas, MVI tiers, dependency scheduling, fallback requirements, and failure-driven replanning.	No PlanForge scheduler, mode selector, or benchmark trace has been implemented here.
verification_bandwidth	Verifier workspace, semantic units, pairwise constraint checks, summary loss, and contradiction pressure as the bottleneck around faster drafting.	The proposed contradiction-rate or constraint-satisfaction tests have not been run.
tokenmana	Token, compute, memory, latency, load, and cognitive-friction budgets through regenerative capacity, bounded pools, burst controls, load signals, and quality/human-friction measures.	No TokenMana simulation, pricing experiment, human study, or local load-stability evidence exists.
talos	Accepted generated output becoming typed artifacts with contracts, evidence records, audit logs, replay, approval gates, isolation, residuals, and delivery discipline.	No Talos runtime, generated-artifact replay, approval-gate test, or security result has been reproduced.
vcm_public	Bounded context packets, source/authority binding, adequacy/admission separation, taint, revocation, invalidation, audit records, and non-success states for generation routes.	Does not prove fast generation quality, speed, or end-to-end model superiority.
spinoza	Proof/citation/procedure-carrying claim graphs, verifier scope, support tiers, contradiction handling, downgrades, and protected axioms separating accepted drafts from truth.	Open-domain autoformalization, verifier completeness, and fast-generation acceptance quality are not solved or reproduced here.
rmi	Generation modes as routable modular capabilities governed by lifecycle, residual escrow, regression floors, readiness, fallback, and promotion discipline.	No empirical fast-generation result, modular-route run, or readiness transition has been reproduced.
ext_speculative_decoding_2022	Speculative decoding as a draft-and-verify family where a cheaper model proposes multiple tokens and the target model verifies accepted candidates.	Reported speedups, proof details, hardware setup, and acceptance-rate results have not been reproduced here.
ext_multi_token_prediction_2024	Multi-token prediction as future-token-head training and proposal mechanism with proposed-token versus accepted-output accounting.	No model was trained or evaluated here; reported benchmark and inference results remain source-reported.
ext_medusa_2024	Internal multi-head drafting with tree-structured candidate verification as an alternative to a separate draft model.	No Medusa heads, branch verifier, training recipe, or speed-quality benchmark was run here.
ext_eagle_2024	Feature-level speculative drafting and target-model verification as a distinct acceleration path from surface-token drafting.	No EAGLE implementation, feature predictor, distribution check, or task evaluation was reproduced here.
ext_lookahead_decoding_2024	Lookahead decoding as exact parallel decoding that trades extra per-step computation for fewer serial decoding steps.	The implementation was not run and its FLOP, latency, branch, or quality behavior was not measured here.
ext_layerskip_2024	Early-exit and self-speculative decoding where early layers draft and later layers verify or correct.	No LayerSkip model, early-exit threshold, or self-speculative benchmark was run locally; same-model verification remains a risk boundary.
ext_pagedattention_vllm_2023	KV-cache and serving-layer acceleration as a memory-management and batching axis separate from single-request verified-output quality.	No vLLM/PagedAttention deployment, serving benchmark, cache isolation audit, or local throughput measurement exists here.
ext_mamba_2023	State-space and recurrent sequence substrates as a backbone-efficiency axis distinct from draft-token acceptance mechanisms.	No Mamba model was trained, served, compared, or routed through ASI Stack governance gates.
ext_llada_2025	Large masked-diffusion language models as an external family that can generate through iterative refinement instead of strict left-to-right autoregression.	No LLaDA run, diffusion benchmark, model-quality test, or ASI Stack integration exists here.
ext_scaling_dllms_2026	Diffusion language-model evaluation by speed-quality tradeoffs rather than perplexity alone.	No diffusion scaling law, checkpoint, code release, sampling schedule, or speed-quality result has been independently inspected or reproduced here.
ext_recurrent_transformer_2026	Layerwise recurrent KV memory, exact tiling, effective-depth/width tradeoffs, cache pressure, and standard autoregressive decoding cost as a current recurrent comparator.	Metadata-first source note only; small-model and asymptotic results are not reproduced and do not establish useful end-to-end speed, quality, recurrent-state correctness, or transfer.

61.7 Mechanism

Fast generation is the time-domain version of compact generation. The system is trying to produce useful verified output with less wall-clock, compute, or memory burden, but the same residual rule applies: proposed output is not accepted output, and accepted output is not useful work until task success, repair cost, fallback, and verification load are visible. CGS gives the generator/verifier frame, PlanForge supplies task-aware mode selection, Verification Bandwidth keeps the verifier bottleneck visible, TokenMana and PagedAttention-style serving work keep memory and load separate from quality, and Benchmaxxing supplies the promotion discipline.

The generation-mode record is therefore a compression receipt for time. It records what the fast path proposed, what the verifier accepted, what work was rejected or repaired, what fallback ran, and whether the useful solution per second improved after the whole route was counted.

The governed lifecycle has eighteen stages:

1. freeze the consumer, permitted use, workload, quality and safety predicates, model and tokenizer, hardware, runtime, serving policy, load, cache state, and horizon;
2. register every route by immutable implementation, configuration, dependency, checkpoint, compiler, and environment identity;
3. preregister optimized quality-equivalent autoregressive, serving, and current-method baselines;
4. bind the exact context packet, prompt/output contract, prefill, cache provenance, warmup, seed, and invalidation policy;
5. classify risk, authority, rights, deadline, reversibility, and minimum verification before selection;
6. define attempted, proposed, accepted, verified, delivered, and useful work as separate denominators;
7. admit only a prospectively eligible mode without hidden-test or outcome peeking;
8. execute with stable request, attempt, branch, seed, batch, tenant, device, and trace identities;
9. measure queueing, prefill, first-token, inter-token, decode, verification, repair, fallback, artifactization, and completion time;
10. record draft acceptance, rejection, correction, divergence, exactness assumptions, and accepted-span length;
11. run task-appropriate claim, code, citation, artifact, policy, safety, and outcome checks;
12. charge verifier, repair, retry, rejection, fallback, escalation, and human review to the candidate route;
13. record compute, occupancy, memory, KV cache, bandwidth, network, storage, energy, money, human, and governance burdens;
14. observe batching, concurrency, tenant isolation, fairness, starvation, cache eviction, and load shedding;
15. route failure or uncertainty to an executable slower fallback, abstention, quarantine, or stop path;
16. estimate causal contribution with matched assignment, negative controls, ablations, load interventions, and complete denominators;
17. allow only consumer-scoped evidence transitions after all preregistered joint gates pass; and
18. monitor expiry, drift, incidents, descendant use, rollback, compensation, and owned residuals after admission.

The operational boundary starts with mode selection around a draft-verify-accept loop.

flowchart LR
  A["Goal and risk tier"] --> B["PlanForge mode selector"]
  C["VCM context packet"] --> B
  D["Latency, compute, memory budget"] --> B
  B -- "select by risk and budget" --> E{"Generation mode"}
  E -- "slow baseline" --> F["Autoregressive baseline"]
  E -- "draft route" --> G["Speculative or MTP draft"]
  E -- "sketch route" --> H["Diffusion or sketch-first pass"]
  E -- "serving route" --> I["Early-exit, state-space, or KV-cache optimized path"]
  E -- "composed route" --> J["Hybrid route"]
  F -- "proposes span" --> K["Draft tokens or artifact span"]
  G -- "proposes span" --> K
  H -- "proposes span" --> K
  I -- "proposes span" --> K
  J -- "proposes span" --> K
  K -- "handoff" --> L["Verifier and acceptance predicate"]

The second boundary decides whether the proposed span becomes useful work, fallback work, or evidence against the route.

flowchart LR
  K["Draft tokens or artifact span"] --> L["Verifier and acceptance predicate"]
  L -- "passes predicate" --> M["Accepted verified output"]
  L -- "fails or partial" --> N["Fallback, repair, or slower mode"]
  M -- "accepted artifact" --> O["Talos artifact"]
  N -- "counts rejected work" --> P["Rejected work + verifier cost"]
  O -- "records accepted work" --> P
  P -- "speed-quality accounting" --> R["Benchmaxxing speed-quality record"]
  R -- "promotion review" --> Q["SCF promotion gate"]

What the generation mode split shows: The selector compares generation modes by task, risk, context, latency, compute, and memory. The acceptance diagram then compares modes by accepted, verified work rather than proposed tokens alone. Rejected spans, verifier cost, fallback, repair, memory pressure, and promotion review stay in the accounting so speed cannot bypass the quality and evidence gates.

The mode selector chooses among autoregressive, speculative, multi-token, diffusion, early-exit, state-space, KV-cache, and hybrid routes by task requirements, risk tier, latency budget, context shape, memory pressure, and verifier availability. The route records proposed output separately from accepted output, sends the VCM packet into the selected generator, routes accepted spans through Spinoza, Talos, Benchmaxxing, and SCF gates, and keeps multi-seed diffusion or hybrid AR/MTP/diffusion ideas as research hypotheses until source-specific experiments or stronger evidence transitions exist.

This makes speed a governed policy choice. A low-risk autocomplete can use a cheap accelerated path. A code patch with tests can use a draft model if the test harness is strong enough. A high-risk claim, authority escalation, or irreversible action can force slower verification-heavy generation even when a faster path exists.

61.8 Generation-mode taxonomy

Mode family	Architectural role	Verification burden	Current book status
Autoregressive baseline	The fallback and measurement baseline for left-to-right generation.	Verify final output, citations, tests, and artifacts according to task risk.	Planned baseline; not run
Multi-token prediction	Draft several future tokens or short continuations from one state.	Count accepted tokens separately from proposed tokens; fallback on rejection.	Source-noted external literature; no local test
Speculative decoding	Cheap model proposes candidate tokens that a stronger verifier accepts or rejects.	Preserve correctness relative to acceptance predicate and count rejected work.	Source-noted external literature; no local test
Multi-head internal drafting	Extra heads propose future tokens or branches without a separate draft model.	Verify branch acceptance and watch backbone/head drift.	Source-noted external literature; no local test
Feature-level or latent drafting	Propose future internal states rather than surface tokens.	Verify that latent proposals preserve output distribution and task success.	Source-noted external literature; no local test
Lookahead and trie retrieval	Retrieve or branch likely continuations from prior patterns.	Verify branches and prevent stale procedural-memory shortcuts.	Lookahead source-noted; trie retrieval remains unmined
Diffusion language modeling	Sketch or denoise many positions in parallel rather than strict left-to-right decoding.	Verify span commitments, uncertainty repair, and final task success.	Source-noted external literature; no local test
Multi-seed diffusion	Run multiple denoising trajectories and select, merge, or repair candidates.	Prove useful solution per second improves after verifier and parallel compute costs.	Speculative research direction
Hybrid AR, MTP, and diffusion	Combine left-to-right drafting, parallel repair, and verifier-guided refinement.	Record route, mode transitions, fallback, and accepted-output accounting.	Speculative research direction
Early exit and self-speculative inference	Use intermediate layers as cheap drafts and deeper layers as verifiers.	Check when early outputs are reliable and when later layers must correct.	Source-noted external literature; no local test
State-space or recurrent alternatives	Reduce sequence-processing cost through different sequence substrate.	Separate cheaper long-context processing from multi-token acceptance.	Source-noted external literature; no local test
KV-cache and serving-layer accelerators	Improve memory layout, batching, and throughput around generation.	Separate aggregate serving throughput from single-request verified output.	Source-noted external literature; no local test

The taxonomy is deliberately plural. Generation governance must not collapse all speed work into one bucket. A decoding trick, a training objective, a serving memory system, and a sequence-model substrate solve different bottlenecks. They can be combined, but their evidence records must stay separate.

61.9 Heterogeneous inference memory: residency, paging, and prediction

A model can be too large for accelerator memory without being too large for the machine. Host RAM, NVMe storage, and remote stores can extend the apparent capacity of a GPU, but they cannot manufacture bandwidth. This is the central fact behind “run a large model on a small GPU” systems: capacity decides whether the bytes can exist somewhere; bandwidth, latency, locality, and overlap decide whether inference is usable.

The first design rule is to name the object being moved. “Model paging” often collapses several objects whose lifetimes and correctness conditions differ:

Physical object	Typical lifetime	What makes reuse valid	Main bottleneck or hazard
Weight shard	checkpoint, adapter, or converted-layout version	exact model lineage, tensor range, dtype, quantization, layout, runtime compatibility, and digest	the same weights may be scanned repeatedly, making storage bandwidth the decode ceiling
KV-cache page	request, prefix, batch, or session	exact model, adapter, tokenizer, token positions, request or sharing scope, precision, and cache epoch	transient state grows with context and concurrency; stale or cross-request reuse changes computation
Activation page	layer, microbatch, or generation step	exact execution state and recomputation contract	spill and reload can cost more than recomputation and can block the next kernel
Expert or neuron block	route, token, or locality epoch	exact expert version plus a valid routing or prediction decision	cold experts create irregular transfers; prediction misses create stalls or approximation
Recurrent state	sequence, segment, or session	architecture-specific state schema, reset rule, and sequence identity	small state can still be latency-critical and cannot be treated as a Transformer KV page
Draft/verifier state	speculative step or accepted span	draft and target identities, acceptance state, and rollback point	rejected proposals can waste compute and evict state needed by the exact target

That distinction separates seven mechanisms that are useful in different regions. Layer streaming loads a predictable layer or shard just before the forward pass needs it. AirLLM is a visible implementation of this idea. Planned heterogeneous placement chooses a steady assignment and transfer schedule across GPU, CPU, and disk; DeepSpeed Inference and FlexGen represent this systems family. Ordinary demand paging fetches an object once execution identifies it. Virtual-memory KV management changes how dynamically growing cache is allocated: PagedAttention uses non-contiguous physical blocks behind a paging abstraction, while vAttention preserves contiguous virtual addresses and maps physical GPU memory on demand. Predictive KV prefetch uses partial computation or a compact importance representation to fetch likely-needed host KV entries before they block a step, as in InfiniGen and SpeCache. Hot/cold sparse placement keeps frequently activated neurons or experts near compute and places colder state in a slower tier, as explored by PowerInfer and related systems. Speculative-decoding-plus-offload uses otherwise idle accelerator capacity for a draft model while target weights are being transferred, as in SpecOffload.

These mechanisms are composable, but they are not synonyms. Predicting a future physical page is speculative paging. Proposing future tokens and checking them with a target model is speculative decoding. Loading the next known dense layer early is deterministic prefetch. Selecting only a predicted subset of KV, neurons, or experts is an approximate policy unless an exact miss path restores the same computation. A compound implementation should report which of these things it does instead of attributing the entire result to “paging.”

61.9.1 The I/O roof under generation

Suppose a dense model requires (W) bytes of weights per full forward pass and the serving path effectively reloads those bytes for each generated token. With sustained usable transfer bandwidth (B), the weight-transfer component alone has the lower bound

[ t_{} , r_{} . ]

This is not a performance prediction. Real time also includes page faults, filesystem and page-cache behavior, small-read penalties, decompression, host-to-device transfer, synchronization, kernel execution, KV traffic, verification, and queueing. It is a fast impossibility check. If the proposed interactive token rate would require several times the measured storage and interconnect bandwidth, no scheduler diagram can rescue it.

Batching changes the arithmetic intensity. One transferred weight block can serve many sequences or tokens before eviction, so FlexGen-style policies can produce useful throughput for latency-insensitive batches even when batch-one interactive decode is slow. The same technique can therefore be good for an overnight corpus job and unacceptable for a conversation. Prefill and decode also differ. Prefill exposes more parallel work and often reuses each loaded weight across many prompt tokens; decode exposes a thin sequential dependency where each next token may trigger another sweep. Every memory-policy result must name batch size, concurrency, prompt length, generated length, cold or warm state, prefix reuse, and whether the reported number is first-token latency, inter-token latency, request throughput, or total completion time.

61.9.2 A physical-memory policy

The runtime needs a versioned Heterogeneous Inference Memory Policy rather than a collection of command-line flags. The policy binds:

• the exact model, tokenizer, adapter, quantization, tensor layout, runtime, kernels, driver, operating system, filesystem, storage device, and hardware;
• weight, KV, activation, expert, recurrent, draft, metadata, and recovery objects to explicit page or shard identities;
• capacity floors and measured bandwidth, latency, concurrency, thermal, energy, endurance, and failure assumptions for every tier;
• placement, eviction, prefetch, prediction, backpressure, fallback, recovery, invalidation, and sanitization behavior; and
• receipts for bytes moved, stalls, unused reads, cache pollution, page faults, prediction misses, compute utilization, latency, throughput, energy, temperature, storage amplification, crashes, and residual state.

An exact residency policy changes where and when state is stored without intentionally changing the mathematical object consumed by the model, apart from declared numerical-layout effects. An approximate memory policy changes the consumed state through quantization, compression, sparse selection, eviction, reconstruction, or skipped fetches. Both can be useful. They need different baselines and claims. A predictor miss in an exact arm must stall, fetch, recompute, or take another explicit exact route; it cannot silently become approximation. An approximate arm must measure task quality and identify the workload region where the loss is accepted.

A cache hit is valid only for the same object version, layout, precision, request or compatible sharing scope, authority, and revocation epoch. This is especially important in multi-tenant and adapter-heavy serving. A page whose bytes are locally present can still be wrong for the request.

61.9.3 Worked request: SSD shards to closure

Consider a quantized checkpoint that does not fit in VRAM or host RAM. Before serving, conversion produces content-addressed layer shards plus a manifest binding their parent checkpoint, tokenizer, adapter, quantization recipe, tensor ranges, layout, and checksums. The worker reserves VRAM for the largest indivisible layer, KV state, kernels, transfer buffers, and recovery headroom. It reserves disk scratch for an interrupted conversion and keeps the original artifact until every transformed shard verifies.

At admission, the runtime freezes the request class: batch one, a bounded prompt and generation length, exact weight streaming, declared KV placement, and a conservative fallback. The first shard moves from SSD to a pinned host buffer and then to the accelerator. While that layer executes, the next shard is prefetched into a different buffer. A successful overlap receipt records both transfer and kernel time; it does not infer that transfer was free. If a checksum fails, storage runs out, the next shard arrives late, thermal throttling changes the deadline, or the request is cancelled, the policy stops or falls back according to the frozen contract.

During decode, the system records bytes read per emitted and per accepted token, page-cache hits, host-to-device traffic, KV growth, stalls, unused prefetch, energy, temperature, and tail latency. If speculative decoding is enabled, its draft and acceptance work is charged separately. Completion closes the request: temporary buffers are invalidated, request-scoped KV pages are released, shared-prefix pages retain only their valid lease, partial outputs and errors become artifacts, and any cache, scratch, corrupted shard, or failed cleanup becomes an owned residual. “The model emitted a token” is therefore an intermediate event, not the end of the memory lifecycle.

61.9.4 Qualification is workload-specific

A competent comparison begins with the smallest fully resident model that meets the task contract, a CPU or unified-memory path, ordinary CPU/GPU offload, layer streaming, planned placement where available, exact and predictive KV variants, architecture-compatible sparse placement, and a prediction-disabled fallback. Compound arms need ablations that distinguish quantization, scheduling, sparsity, paging, and speculative decoding.

The joint report should include task success and quality alongside median and tail first-token latency, inter-token latency, end-to-end time, request and token throughput, peak VRAM and RAM, bytes read and written per token, transfer overlap, misses, unused prefetch, cache pollution, utilization, energy, temperature, storage wear proxy, conversion amplification, interruption, and recovery. A route qualifies only for the region where it meets the requested quality, latency or throughput, memory, stability, and recovery contract.

The current literature is a design map, not a local result. AirLLM, Accelerate, llama.cpp, and related tools show that disk- and host-backed execution is possible. FlexGen shows why batching and planned placement matter. LLM in a Flash and PowerInfer show how locality can reduce transferred bytes. PagedAttention and vAttention preserve a live layout choice rather than a settled universal answer. InfiniGen and SpeCache motivate predictive KV fetch; SpecOffload motivates composition with token speculation; ATSInfer adds a recent tensor-granular consumer-device comparator. None of those sources proves that this repository has a fast, exact, durable, or generally superior implementation. They tell us which mechanism distinctions a serious implementation and benchmark must preserve.

61.10 Inference reuse: prefix hits, KV reuse, and response caching

“Cache hit” is too vague for an inference contract. It can mean that one autoregressive request reused its own earlier keys and values, that a later request inherited an exact prompt prefix, that a worker fetched compatible KV state from another tier, or that the service returned an old answer without running the model at all. Those routes avoid different work and carry different correctness, privacy, freshness, and pricing obligations.

Cache object	Match rule	Work avoided	Work that remains
In-request decode KV	Same live sequence and model state	Recomputing prior-token keys and values	Every next-token model step
Exact prompt-prefix KV	Exact compatible token prefix	Prefill for the reusable prefix	Uncached suffix prefill and a new output
Prompt module	Declared module, position schema, and composition	Attention-state work for the module	Composition, suffix, and new output
Persistent or disaggregated KV	Compatible state available on another tier or worker	Repeated prefill after retrieval	Lookup, transfer, validation, suffix, and decode
Non-prefix chunk KV	Compatible chunks plus a recomputation policy	Part of repeated chunk prefill	Cross-context repair and decode
Exact output memo	Exact request and dependency closure	The model request	Validity checks, disclosure, and downstream verification
Semantic response cache	Similar request accepted by an approximate policy	The model request on a hit	Similarity lookup, freshness checks, and often verification

The first line is ordinary autoregressive execution, not a cross-request hit. The next four reuse model computation but still produce a new answer. The last two reuse a decision or artifact. A service that bills “cached inference” should say which row occurred.

61.10.1 Exact prefix reuse saves prefill, not generation

During prefill, a causal transformer computes attention state for the prompt. When a later request begins with the same compatible token sequence, the runtime can retain or load that prefix state and compute only the uncached suffix. This principally reduces prefill work, time to first token, and prefill capacity pressure. It does not remove the sequential decode of newly generated tokens. It also does not guarantee that the new answer matches an earlier answer: sampling, an uncached suffix, tool state, model behavior, or downstream verification can all differ.

Current OpenAI, Anthropic, and Gemini documentation exposes variations of this contract. The details differ—automatic versus explicit caching, breakpoints, minimum sizes, lifetimes, write meters, read meters, and storage charges—but the common rule is to put the largest stable prefix before volatile material. Stable tool definitions, system policy, output schemas, examples, and canonical reference material are natural candidates. Per-user data, timestamps, retrieved material that changes frequently, and the current question usually come later. That layout is subordinate to instruction and security semantics: untrusted text must not be moved into a privileged position merely to win a cache hit.

Text equality is not enough to establish state equality. A reusable-prefix key may need to bind the provider, model and checkpoint, tokenizer and vocabulary, adapter, quantization, runtime, kernels, precision, position scheme, prompt tokens and order, image bytes and detail, tools, output schema, and any reasoning or generation configuration that changes the cached computation. It also needs a tenant or sharing scope, authority and policy epoch, privacy and rights class, source versions, cache format, block chain, creation time, expiry, and invalidation epoch. Where a provider keeps part of that identity opaque, the application should record both the fields it supplied and the provider-reported cached-token count instead of claiming that it verified the internal key.

vLLM’s Automatic Prefix Caching makes the systems form concrete: completed KV blocks are chained through parent and token-block hashes, adapter and multi-modal identity can enter the hash, and unreferenced cached blocks are evicted under pressure. Its cache-salt option matters because shared-cache timing can itself reveal that another trust domain requested a prefix. A cryptographic digest reduces accidental or adversarial collision risk, but it does not supply authorization, freshness, deletion, or useful economics.

61.10.2 Reuse is a scheduling and placement policy

SGLang’s RadixAttention treats shared prefixes as workload structure. Requests or branches with a long shared prefix can occupy a radix tree, and a cache-aware scheduler can order work to preserve useful state. That expands the decision from “is this block present?” to “which admitted request should run next?” Prefix locality can improve aggregate throughput while worsening a deadline, starving an unrelated request, or creating cross-tenant coupling. The scheduler therefore needs explicit fairness, risk, deadline, isolation, backpressure, and tail-latency constraints in addition to a reuse objective.

Prompt Cache takes a more explicit route. It defines reusable prompt modules through a schema and preserves their positional meaning when composing a request. This can make reuse auditable: the caller names a versioned module rather than hoping an opaque prefix happens to match. It also creates a new contract. The module schema, positions, composition rules, source dependencies, and rights must remain compatible at use time.

Mooncake pushes KV state beyond one worker, treating DRAM, SSD, network links, prefill workers, and decode workers as a distributed cache fabric. Persistent or disaggregated reuse is beneficial only when

[ t_{} + t_{} + t_{} < t_{} + _{} ]

under the same quality and lifecycle contract. A remote KV object can be numerically compatible yet slower to fetch than recompute, expired under a source correction, inaccessible to the current tenant, or costly to delete from every replica. Cross-worker reuse is distributed state management, not a free extension of a local allocator.

61.10.3 Non-prefix fusion needs recomputation or an approximation claim

Retrieved documents and reusable instruction blocks do not always remain one unchanged prefix. An independently cached chunk was encoded at some position with some preceding context. Concatenating several such KV objects can omit the cross-attention that full prefill would have computed between the chunks. CacheBlend makes this failure explicit and selectively recomputes a subset of token state while overlapping that work with KV retrieval.

The important lesson is not that selective recomputation is always enough. It is that arbitrary chunk reuse is not exact prefix reuse. Unless full recomputation or an accepted equivalence argument restores the same computation, the route is approximate. Qualification needs a full-prefill baseline, adversarial chunk order and interaction cases, task quality and calibration, recomputation selection diagnostics, and a method-specific rescue ladder. A fast negative result from naïvely concatenated caches would only refute that broken composition, not non-prefix reuse as a class.

61.10.4 Output and semantic caches reuse decisions

An exact output cache returns a prior answer. Its key must close over every dependency that can change the answer’s admissibility: request and model identity, decoding policy and randomness, retrieved sources, tool results, current time and locale, user permissions, policy, external side-effect state, and output schema. That closure can be useful for immutable classification, content-addressed transformation, or verified deterministic compilation. It is much narrower for personalized, dynamic, open-world, or side-effecting work.

A semantic response cache goes further by treating embedding similarity as permission to reuse a prior answer. Microsoft’s Azure policy documentation explicitly warns that a semantic hit can be incorrect, outdated, or unsafe. This is approximate decision reuse, not numerical acceleration. It should be a named route with calibrated thresholds, tenant and policy partitions, risk admission, stale and poisoned-entry tests, disclosure, verification where required, and a fresh-model fallback. Time-sensitive facts, personalized decisions, high-impact advice, security operations, and rights-sensitive material should default to fresh evaluation unless a claim-specific policy earns a narrower exception.

61.10.5 The cache-reuse receipt

Every route returns a CacheReuseReceipt even on a miss. The minimum record binds:

• cache kind, entry identity, key digest, matched-prefix and total-input lengths, hit, miss, partial, bypass, or invalid state, and reason;
• model/runtime identity, tenant and sharing scope, dependency digest, creation and access times, lifetime, invalidation epoch, and deletion state;
• lookup, hash, write, read, transfer, storage-time, eviction, recomputation, verification, output, and provider-reported token meters;
• cold or warm class, queue time, time to first token, time per output token, completion time, accepted-output state, repair, and fallback; and
• authority decision, privacy treatment, reuse disclosure, cleanup, and any unresolved residual.

The matched benchmark uses the same model, prompt, hardware, runtime, load, tenant mix, output work, validation, and quality bar with the cache cold or disabled. It reports useful-token and byte reuse, not only request hit rate; cold, warm, steady, burst, eviction, failover, and adversarial-tenant regimes; and quality, safety, tails, fairness, storage, transfer, energy, money, invalidation lag, and deletion closure. Before a negative inference, key correctness, prompt layout, capacity, lifetime, scheduling, prewarming, and the source-recommended configuration must pass self-tests and rescue. The current sources define this design space. No cache implementation or speed result has been reproduced here.

61.10.6 Precision routing is an execution policy

Static quantization, layer-wise mixed precision, progressive refinement, and dynamic precision are distinct generation routes. A request-level policy may start with a qualified base representation and request residual precision when observable uncertainty, disagreement, consequence, or verification failure crosses a threshold. The terminal alternatives remain the full reference route, abstention, or escalation.

The router cannot inspect the answer key or use a confidence signal that was never calibrated for the transformed model. Its inputs, thresholds, maximum refinement, oscillation control, latency deadline, fallback capacity, and adversarial tests are frozen with the route. If the low-precision path changes the uncertainty signal used to decide whether more precision is needed, that feedback loop receives explicit positive controls.

The base route has a named safe operating envelope rather than a global “2-bit works” label. Envelope predicates can include domain membership, decision margin, activation and accumulator range, agreement among cheap replicas, protected-topic detection, a local verifier result, and calibrated route confidence. Precision escalation is then a metareasoning purchase: request another level only when its expected protected-behavior value exceeds its complete marginal cost, subject to hard clauses that cannot be traded for latency. Weight, activation, accumulator, KV-cache, and context precision are separate resources; retrieval, more samples, a longer trace, a specialist, an exact tool, or human review may be the cheaper qualified substitute.

The matched comparison reports accepted output quality and protected slices beside time to first token, decode rate, tail latency, throughput, bytes moved, energy, residual-plane frequency, fallback frequency, router overhead, and verification cost. A dynamic policy is faster only if the complete workload distribution benefits after those costs. A strong tuned static mixed-precision policy is the minimum serious baseline.

flowchart LR
    A["Admitted request"] --> B["Base precision route"]
    B --> S["Uncertainty / consequence / verifier signals"]
    S --> D{"Contract likely satisfied?"}
    D -->|yes| O["Accept bounded output"]
    D -->|refine| P["Load residual precision"]
    P --> S
    D -->|unknown or deadline| F["Reference / abstain / escalate"]

61.10.7 Dual-vocabulary generation separates reasoning from realization

KERC proposes three logical code spaces: a rich surface vocabulary for input and output, a smaller canonical Kernel vocabulary for recurrent reasoning, and a pointer/control space for objects, handles, residuals, versions, and byte fallback. The proposed speedup does not come from calling the surface language unimportant. It comes from avoiding a large surface projection and its lexical competition at every internal step while retaining a separate high-fidelity realization path at the end.

The cleanest architecture uses a surface compiler, a large core reasoner over Kernel plus pointers, and a surface renderer with copy access to protected objects. A lower-parameter alternative shares a trunk while retaining distinct surface, Kernel, and copy heads. Full vocabulary tying is not assumed: surface and Kernel forms may share a stable concept base while adding code-specific or form-specific components. The shared-trunk design saves parameters but raises interference risk; the modular design simplifies replacement and multilingual rendering but can make compiler and renderer passes dominate latency.

The output-head hypothesis is workload-specific. If a long internal trace has L_K Kernel steps and the final answer has L_Y surface steps, the large surface head is needed for roughly L_Y rather than L_K + L_Y positions. That arithmetic matters only after charging compilation, residual coding, object retrieval, rendering, recompilation, local regeneration, and fallback. A small core surrounded by two large slow language models is not a fast generation architecture.

The representation boundary is compatible with byte-level dynamic chunking at ingress and with latent recurrent intervals inside the core. Discrete Kernel checkpoints can be emitted when a tool call, memory write, branch, verification, inter-agent handoff, or audit view needs a stable object, while unobserved microsteps remain latent. These hybrids are alternatives to test, not free improvements to stack together. No source result shows that a dual-vocabulary implementation beats a strong ordinary, byte-level, dynamic chunking, latent, or copy-aware baseline.

61.11 Interfaces

Generation choices are surfaced through a generation-mode record. Fast Generation sits between route eligibility and delivered work: it accepts a bounded request, obtains context and runtime identity, and returns proposals, verification outcomes, fallback activity, costs, residuals, and closure receipts. Each neighboring layer retains its own authority. Speed-route admission cannot grant data rights, declare evidence, erase failures, redefine quality, or turn generated text into an executed effect. That separation lets future decoding substrates change without rewriting the surrounding governance and accounting contracts.

Boundary owner	Fast Generation handoff
Intent and Planning	Exact request, consumer, task, risk, deadline, alternatives, and fallback objective; no generation authority.
Virtual Context	Versioned model-visible packet, omission and source lineage, cache eligibility, adequacy, expiry, and invalidation.
Routing and Readiness	Eligible candidate modes, qualifications, capacity, leases, and abstention; Fast Generation retains request-specific admission.
Runtime Adapters	Model, tokenizer, checkpoint, compiler, device, engine, batch, concurrency, seed, clock, and instrumentation identity plus lease enforcement.
Resource Economics	Prospective budgets and the complete realized latency, compute, memory, bandwidth, energy, money, human, and governance bill.
Security, Authority, and Rights	Data, tenant, effect, disclosure, approval, reversibility, and affected-party constraints independent of speed.
Generator	Attempt, branch, proposal, acceptance, correction, cancellation, and resource traces rather than only a final string.
Verification	Exact candidates and obligations in; bounded verdict, dependence, disagreement, cost, ignorance, and residual records out.
Labor OS, Talos, and Artifact Graphs	Accepted output becomes a typed delivered artifact with job, effect, provenance, replay, and terminal receipt custody.
Benchmaxxing	Complete attempt universes become matched quality, usefulness, latency, resource, tail, fairness, and uncertainty comparisons.
Claim Ledgers and Readiness Gates	Evidence transition, qualification, expiry, downgrade, quarantine, and no-change decisions; no controller self-promotion.
Operators and consumers	Monitoring, rollback, compensation, appeal, residual, requalification, slower-route, and stop controls.

Minimum fields:

• task_id
• risk_tier
• latency_budget
• compute_budget
• memory_budget
• context_packet_id
• generation_mode
• draft_source
• verifier
• acceptance_predicate
• proposed_tokens_or_spans
• accepted_tokens_or_spans
• wall_clock_time
• quality_or_pass_result
• repair_or_fallback
• promotion_decision
• non_claims

A generation-mode record is boring by design. It prevents a fast-generation chapter from becoming a pile of impressive methods without a route, verifier, fallback, or promotion rule.

It also gives consumers a refusal path. A high-risk task can require stricter verification or disallow a fast mode; a low-risk draft can accept provisional output with explicit fallback. Without that consumer policy, speed becomes a hidden pressure to skip checks.

61.12 Invariants

The invariants bind speed to governance.

1. Every speed claim binds an exact consumer, use, workload, model/tokenizer, hardware/runtime, serving policy, load, cache state, metric, and time window.
2. Workload, baselines, gates, accounting, statistical plan, stop rule, and promotion ceiling are frozen before results.
3. Immutable model, draft, verifier, code, dependency, compiler, hardware, configuration, seed, clock, and artifact identities support replay.
4. Strong quality-equivalent baselines run under matched inputs, resources, evaluators, warmup, cache, batch, concurrency, retries, and horizon.
5. Attempted, proposed, accepted, verified, delivered, and useful output never substitute for one another.
6. End-to-end latency includes queueing through delivery; decoder-only time cannot support an end-to-end claim.
7. Verification, repair, fallback, review, and disagreement remain charged to the candidate even when they erase its gain.
8. Warm caches, prefix reuse, compilation, batching, and startup amortization are declared and matched.
9. Throughput, first-token, inter-token, completion, deadline, median, and tail metrics remain separate.
10. Compute, memory, KV cache, bandwidth, energy, money, human and governance work, displaced capacity, and opportunity cost remain visible.
11. Failures, timeouts, cancellations, OOMs, rejected drafts, retries, abstentions, fallbacks, and censored attempts remain in the denominator.
12. Fallback is executable, capacity-reserved, quality-checked, and included in latency and cost.
13. High-risk, authority-changing, rights-sensitive, or irreversible work may prohibit accelerated generation or require a slower independent verifier.
14. Shared model, data, cache, prompt, or failure modes preclude an independence claim without tested dependence evidence.
15. Batching, tenant mix, eviction, load shedding, starvation, and fairness are reported beside aggregate gains.
16. Transfer across workload, model, tokenizer, hardware, runtime, quantization, context, organization, rights regime, or time requires a test.
17. Qualification expires after material change and can be downgraded immediately by contrary evidence.
18. No raw rate, FLOP estimate, theoretical parallelism, external report, synthetic template, import, Lean theorem, validator, or throughput number alone promotes the core.

These invariants are intentionally conservative. They do not forbid fast decoding. They prevent fast decoding from laundering unverified output into the stack as if it were a proven capability.

Acceptance logs must retain rejected drafts and repair attempts when they explain latency, quality, or residual risk.

61.13 Failure modes

Fast generation fails when speed is measured at the wrong boundary.

1. Proposed tokens replace accepted, verified, delivered, or useful work in the numerator.
2. The candidate is compared with an unoptimized, lower-quality, smaller-context, differently quantized, or otherwise weaker baseline.
3. The task cohort is cherry-picked for high acceptance, short outputs, repeated prefixes, easy checks, or favorable occupancy.
4. Average quality hides rare catastrophic, citation, code, policy, rights, multilingual, long-context, or high-risk regressions.
5. A slow, weak, correlated, overloaded, or proxy-scored verifier fails to preserve the claimed quality boundary.
6. Repair, retry, rejection, fallback, or human-review burden is omitted until the apparent advantage disappears.
7. Warm caches, prefixes, compiled kernels, startup amortization, or precomputed candidates benefit only the fast route.
8. Batch throughput is reported as request speed, or first-token latency hides worse completion and deadline behavior.
9. Batch, tenant, or scheduler choices improve averages by starving low-volume, long-context, accessibility, or high-priority work.
10. KV pressure, memory bandwidth, network, energy, thermal throttling, or parallel sampling erases gains outside the headline point.
11. Draft heads, latent states, diffusion sketches, tries, recurrence, or caches drift after model, data, prompt, or context change.
12. Multi-seed or hybrid generation multiplies candidates without improving verified useful work per time and total burden.
13. Latency pressure weakens risk, authority, safety, or rights gates.
14. Fallback is missing, capacity-starved, late, inconsistent, unable to recover partial state, or untested under failure.
15. Clock choice, synchronization, instrumentation, queue exclusion, early stopping, or censoring biases the comparison.
16. Failures, cancellations, OOMs, disagreements, and null seeds disappear from the denominator or publication surface.
17. A result is generalized across models, workloads, contexts, hardware, runtimes, engines, load, organizations, or time without transfer evidence.
18. Stale qualification remains routable after model, verifier, tokenizer, quantization, compiler, hardware, cache, workload, threat, or rights change.

The multi-seed case is especially important for this book. Multiple candidate trajectories can improve robustness only if selection, verification, and parallel hardware costs are cheap enough to beat a single stronger route. Otherwise, multi-seed generation is just a more expensive way to produce more candidates. The book should keep it as an original research direction, not as an established capability.

61.14 Minimum Viable Implementation

The current minimum is much narrower than a decoder benchmark: one not-run generation-mode fixture; one bounded Project Theseus import; one validator-replay probe; one deterministic four-task accounting bundle; one selected support-lane aggregate; two accepted no-change decisions; three retained finite countermodels; and a seventeen-declaration, eight-stage, sixty-route request-to-closure refinement. Its independent consumer rejects 51/51 non-accepting mutations and digest-binds the exact baseline, task-bundle, and Theseus suites. Thirty-five weaker baseline declarations are retired. No language model, accelerated decoder, live serving engine, model-output verifier, route selector, latency clock, hardware meter, or deployed fallback runtime ran.

The current generation_mode_record fixture validates only the record shape: task, risk tier, budgets, context packet, mode, draft source, verifier, acceptance predicate, output accounting fields, promotion decision, measurement status, metric definitions, evidence references, and non-claims. It intentionally records not_run, not measured, and not_evaluated for the demo route. The Lean envelope adds bounded rejection cases for promotion candidates that omit accepted-output or verifier-cost evidence, failed accelerated drafts that keep promotion open without fallback or residual handling, and high-risk fast-mode selections that omit verifier, risk-override, or slower-fallback records.

The Project Theseus generation-mode import adds the first public-safe implementation-reference slice for this boundary. The imported static gate summary records 18 modes, 13 comparisons, zero hard gaps, zero modes with missing report refs, five hard boundary gates passing, and zero promotable comparisons. Five comparisons saw accepted-span speed lift, but every candidate task-pass count stayed at zero and useful solution per second stayed at zero, so the import functions as negative evidence against raw-throughput promotion rather than as a speedup result. The digest-bound Python validator owns those exact report counts; Lean now owns the general reachable mode, fallback, raw-speed, promotion, accounting, and closure rules in AsiStackProofs.FastGenerationRefinement. That separation removes the old hand-copied summary bridge while preserving rejection of hard boundary-gate failure, missing-report-ref overclaim, raw-speed promotion, private-payload copying, and useful-speed overclaim. The Theseus support replay probe reruns the ASI-side generation-mode validator and the connected architecture-gate validator, then records command-output and artifact digests. It does not copy private task rows, candidate outputs, prompts, training rows, checkpoints, traces, or benchmark payloads, and it does not rerun Theseus.

The Fast generation task bundle adds the first public-safe task-bundle accounting replay inside this repository. Result fast_generation_task_bundle_2026_07_02_local compares route://autoregressive-reference, route://fast-template-verified, and route://latency-only-proxy across four deterministic receipt tasks. The verified fast-template route passes the same 4/4 tasks as the baseline at 264 deterministic cost units instead of 632, while the cheaper latency-only proxy is rejected at 176 cost units because it fails verifier, fallback, residual, and support-state boundaries. The accepted no-promotion decision at evidence_transitions/v1_x_measured/fast_generation_task_bundle_no_change.json keeps that slice at argument and blocks model-speed, useful-solution-per-second model performance, serving-throughput, route-selector adequacy, benchmark, model-quality, deployed-routing, and chapter-core promotion claims. This is a no model-speed or deployment claim: it records route-accounting discipline and useful-solution-per-cost behavior for a local deterministic bundle, not an AI model useful-solution-per-second improvement, reproduced speculative decoder, MTP run, diffusion benchmark, KV-cache result, serving throughput result, or support-state promotion.

The selected support-lane aggregate adds drift detection across the two active support chapters. theseus-fast-support-lane-2026-07-03-local reruns the Theseus generation-mode import validator, the Theseus support replay probe, the Theseus public task-bundle import validator, and the Fast Generation task-bundle validator, then records 16 tracked artifact digests, 68 public task records, 14 expected-invalid or rejected controls, and two accepted no-promotion decisions. Its copied Lean summary mirror was retired after semantic audit because the executable aggregate already owns those facts. Its boundary remains narrower than a capability result: no clean live Project Theseus replay, no model quality result, no generation-speed result, no useful-solution-per-second model-performance result, and no chapter-core promotion.

The registry should allow experimental entries such as multi_seed_diffusion or hybrid_ar_diffusion_repair, but those entries must remain unroutable or experimental until source notes and tests exist.

61.15 Mature Research Target

The mature target is a governed generation market tested on preregistered natural chat, code, retrieval, extraction, long-context, multilingual, accessibility, agentic, artifact-producing, and high-risk workloads plus adversarial quality, cache, load, verifier, drift, rights, and failure cases. It compares optimized target-model autoregression, ordinary serving, speculative decoding, multi-token and multi-head drafting, feature-level drafting, lookahead, early exit, recurrent or state-space backbones, KV-cache and batching systems, diffusion, hybrid routes, conservative slow verification, and the full controller under matched models, inputs, hardware, runtime, load, cache, evaluators, resources, retries, and time.

The joint decision surface includes useful accepted success, exactness or quality, unsafe acceptance, false rejection, coverage and abstention, first-token, inter-token, completion, and deadline latency, throughput, median and tail behavior, compute, memory, KV cache, bandwidth, energy, money, human and governance work, fairness, rights, fallback, recovery, residuals, and total lifecycle burden. No single rate can compensate for a failed safety, rights, quality, or recovery gate.

Prospectively fixed interventions remove mode selection, accepted-output accounting, verifier independence, cache matching, queue-to-delivery timing, repair attribution, fallback, resource accounting, high-risk override, load isolation, or expiry one at a time. Predicted benefits must disappear or the claimed mechanism narrows. Negative, null, censored, failed, and contrary outcomes remain in the attempt universe.

Independent generator, verifier, evaluator, serving, and measurement implementations must reproduce the conclusions. Transfer must vary models, workloads, languages, context lengths, hardware, runtimes, organizations, rights regimes, attacks, load states, failures, updates, and time. A route qualifies only for the exact cells it survives and expires after material change.

This mature campaign is a target, not a current result. The static import, deterministic template bundle, finite proofs, and source-reported systems do not establish model speed, serving throughput, useful-solution-per-second improvement, quality preservation, generality, deployment readiness, or state-of-the-art performance.

61.16 Codex test plan

Test	Purpose	Status
Generation-mode record fixture validation	Validate that a route record names task, risk tier, budgets, context packet, mode, draft source, verifier, acceptance predicate, accounting fields, metric definitions, evidence refs, and non-claims.	implemented; passing via python3 scripts/validate_protocol_examples.py
Acceleration accounting negative case	Prove that a finite promotion candidate is rejected when accepted-output or verifier-cost evidence is missing.	implemented in AsiStackProofs.FastGeneration.promotion_candidate_missing_accepted_output_or_verifier_cost_rejected; no decoder benchmark or quality result promoted
Failed-acceptance handling negative case	Prove that a finite failed accelerated draft cannot satisfy the handling predicate when fallback, residual recording, or promotion blocking is absent.	implemented in AsiStackProofs.FastGeneration.failed_acceptance_without_fallback_or_residual_rejected; no fallback runtime executed
High-risk fast-mode review negative case	Prove that a finite high-risk fast-mode selection is rejected when verifier, risk-override, or slower-fallback records are missing.	implemented in AsiStackProofs.FastGeneration.high_risk_fast_mode_without_verifier_or_override_rejected; no risk classifier or mode selector executed
Fast Generation request-to-closure refinement	Independently consume every route in the eight-stage lifecycle, reject identity, admission, verification, fallback, accounting, promotion, support, effect, and closure failures, and bind the exact source suites without copying their conclusions into Lean.	implemented; python3 scripts/validate_fast_generation_refinement.py reaches all 60 routes, rejects 51/51 non-accepting mutations, and digest-binds the 2-valid/4-invalid baseline, three-route/four-task, and 1-valid/6-invalid Theseus suites; support/effect none; no model-speed, useful-throughput, deployed-fallback, serving, transfer, or SOTA claim
Project Theseus generation-mode import validation	Validate a public-safe static Theseus generation-mode gate summary with 18 modes, 13 comparisons, zero hard gaps, zero modes with missing report refs, zero promotable comparisons, digest-bound Lean policy alignment, and mutation controls for hard boundary-gate failure, private payloads, missing-report-ref overclaim, support overclaim, raw-speed promotion, and useful-speed overclaim.	implemented; passing via python3 scripts/validate_theseus_generation_mode_import.py; the external validator owns exact report counts and AsiStackProofs.FastGenerationRefinement owns reachable policy; no live Theseus replay, generation-speed result, useful-solution-per-second win, or support-state promotion
Theseus support replay probe	Replay the Project Theseus generation-mode import validator together with the architecture-gate import validator and check command-output digests, tracked artifact hashes, and no-transition boundaries.	implemented; passing via python3 scripts/validate_theseus_support_replay_probe.py; no live Theseus replay, public task bundle, speed-quality result, useful-solution-per-second win, or support-state promotion
Fast generation public-safe task bundle	Replay a local public-safe task bundle with baseline, verified fast-template candidate, latency-only negative control, quality checks, verifier/fallback/residual records, and deterministic useful-solution-per-cost accounting.	implemented; passing via python3 scripts/validate_fast_generation_task_bundle.py; result fast_generation_task_bundle_2026_07_02_local; selected route://fast-template-verified; rejected route://latency-only-proxy; accepted no-promotion decision evidence_transitions/v1_x_measured/fast_generation_task_bundle_no_change.json; no model-speed or deployment claim, no useful-solution-per-second model claim, or support-state promotion
Theseus/Fast selected support-lane aggregate	Replay the selected Theseus/Fast support validators together and check command-output digests, tracked artifact hashes, public task counts, rejected-control counts, and accepted no-promotion decisions.	implemented; passing via python3 scripts/validate_theseus_fast_support_lane.py; result theseus-fast-support-lane-2026-07-03-local; copied formal mirror retired; 68 public task records and 14 expected-invalid or rejected controls; does not prove clean live Project Theseus replay, model quality, generation speed, or support-state promotion; does not promote any chapter core claim
Autoregressive baseline speed-quality test	Establish the baseline wall-clock, accepted-output, and task-success measurements.	planned; not run
Speculative decoding acceptance test	Measure proposed tokens, accepted tokens, rejected work, verifier cost, and output correctness under an explicit acceptance predicate.	planned; not run
Multi-token draft-head acceptance test	Test whether k-token proposals improve accepted output per second after rejection and fallback are counted.	planned; not run
Medusa-style internal-head comparison test	Compare internal multi-head proposals against external draft-model proposals under the same verifier.	planned; not run
Diffusion small-model speed-quality curve	Measure whether parallel denoising improves final task success per wall-clock second.	planned; not run
Multi-seed diffusion useful-solution-per-second test	Test whether multiple denoising trajectories improve pass rate or verified success after parallel compute and selection costs.	planned; not run
Hybrid AR-to-diffusion repair test	Test whether generating a left-to-right draft and repairing spans beats full regeneration.	planned; not run
Planner-selected generation-mode routing test	Compare task-aware mode selection against one fixed decoding mode.	planned; not run
Risk-tiered decoding enforcement test	Ensure high-risk tasks route to slower verified modes when policy requires it.	planned; not run
KV-cache throughput accounting test	Separate aggregate serving throughput, memory pressure, and single-request verified output.	planned; not run

The implemented rows validate record discipline, a reachable finite request-to-closure policy, every modeled route, 51 rejecting mutations, three exact digest-bound source suites, one static Theseus import boundary, and a local replay probe over the ASI-side Theseus validators. The remaining rows are a backlog for turning the design rationale into empirical capability evidence.

61.17 Formalization hooks

Tag	Module	Target	Status
lean:fast_generation.mode_selection.operational_invariant	AsiStackProofs.FastGenerationRefinement	Reachable admission binds task, context, consumer, mode, risk, verifier, baseline, quality, latency, resource, fallback, and rights records before fast selection.	implemented
lean:fast_generation.verified_speed.failure_blocks_promotion	AsiStackProofs.FastGenerationRefinement	Reachable accounting and decision routes block raw-speed promotion without accepted output, task success, matched baseline, complete costs, and an evidence transition.	implemented
lean:fast_generation.mode_admission_lifecycle_route	AsiStackProofs.FastGenerationRefinement	Eight stages and sixty independently consumed routes govern context binding, selection, drafting, verification or fallback, useful-outcome accounting, decision, and closure.	implemented
lean:fast_generation.theseus_import_fixture_bridge	AsiStackProofs.FastGenerationRefinement	A digest-bound external validator owns Theseus report counts while Lean owns general reachable raw-speed, fallback, promotion, and closure policy.	implemented
lean:fast_generation.task_bundle_fixture_bridge	AsiStackProofs.FastGenerationRefinement	A digest-bound external validator owns the three-route task result while Lean owns general task-success, cost-separation, speed-proxy, support-transition, and closure policy.	implemented

AsiStackProofs.FastGeneration now retains only three genuine finite countermodels. AsiStackProofs.FastGenerationRefinement contains seventeen declarations over eight reachable stages and sixty routes. The independently implemented consumer reaches all sixty routes, rejects all 51 non-accepting mutations, reruns the exact baseline, task-bundle, and Theseus validators, and binds their artifacts by SHA-256. Lean governs the general route policy; external validators govern source-result counts. The verified witness closes with one authored useful-outcome accounting event, while the fallback witness closes only after fallback is activated and costed. Neither witness assigns support or external-effect authority. The cross-chapter support aggregate is intentionally executable-only; its former copied AsiStackProofs.TheseusReference fixture mirror is retired.

Thirty-five frozen baseline declarations were physically retired because they restated assumptions, projected fields, or copied fixture summaries. The remaining twenty live family declarations are three countermodels and seventeen refinement theorems. Compilation and route coverage prove only this finite authored lifecycle under trusted packet fields. Authored taskSuccess, usefulDenominator, verificationPassed, cost, and fallback fields are not measurements; the model does not establish decoder speed, verifier correctness or independence, route-selector adequacy, deployed fallback, natural-workload usefulness, serving behavior, transfer, or SOTA.

It does not prove model generation speed, useful throughput, verifier adequacy, deployed fallback, serving performance, transfer, SOTA, or any chapter-core support movement.

61.18 Source crosswalk

Source ID	Planned use	Readiness
cgs	Connect fast generation to compact generative systems, residual burden, and the generator/verifier loop.	source note available; claim-level mapping recorded
cognitive_loop_closure	Frame repeated generation and chunking as procedural-memory pressure.	source note available; claim-level mapping recorded
benchmaxxing	Define the measurement discipline for useful output, regression, and promotion.	source note available; claim-level mapping recorded
planforge	Route generation mode selection through planning and task constraints.	source note available; claim-level mapping recorded
verification_bandwidth	Keep verifier capacity visible as the possible bottleneck.	source note available; claim-level mapping recorded
tokenmana	Account for token, compute, memory, and latency budgets.	source note available; claim-level mapping recorded
talos	Turn accepted generated output into artifacts rather than loose text.	source note available; claim-level mapping recorded
vcm_public	Supply bounded context packets and context-loss contracts to the selected generation mode.	source note available; claim-level mapping recorded
spinoza	Verify, critique, repair, or reject generated claims and candidates.	source note available; claim-level mapping recorded
rmi	Connect generation modes to modular routing, readiness, and fallback.	source note available; claim-level mapping recorded
ext_speculative_decoding_2022	Source-note speculative draft-and-verify decoding.	source note available; reported results not reproduced
ext_multi_token_prediction_2024	Source-note future-token-head training and proposal mechanisms.	source note available; reported results not reproduced
ext_medusa_2024	Source-note internal multi-head/tree-branch drafting.	source note available; reported results not reproduced
ext_eagle_2024	Source-note feature-level speculative drafting.	source note available; reported results not reproduced
ext_lookahead_decoding_2024	Source-note exact lookahead/parallel decoding.	source note available; reported results not reproduced
ext_layerskip_2024	Source-note early-exit and self-speculative decoding.	source note available; reported results not reproduced
ext_pagedattention_vllm_2023	Source-note KV-cache memory management and serving throughput acceleration.	source note available; reported results not reproduced
ext_mamba_2023	Source-note state-space/recurrent sequence substrates.	source note available; reported results not reproduced
ext_llada_2025	Source-note large masked-diffusion language modeling.	source note available; reported results not reproduced
ext_scaling_dllms_2026	Source-note diffusion LLM speed-quality evaluation.	source note available; reported results not reproduced
ext_recurrent_transformer_2026	Current comparator for layerwise recurrent KV memory, exact tiling, effective depth, cache pressure, and standard autoregressive decoding cost.	source-note mapping recorded; small-model and asymptotic results not reproduced

The external literature queue is now partially mined for the selected primary papers. It is still not complete: code artifacts, exact experimental sections, later variants, and implementation-specific claims must be inspected before the book can summarize reported speedups in detail or promote support beyond argument.

61.18.1 Manifest source assignment reconciliation

These rows keep Fast Generation Architectures’s manifest assignments visible at their recorded review boundary. Passage review does not establish local reproduction, performance, safety, deployment, or support-state movement.

Source	Intake role	Boundary
kernel_english_residual_compiler	Metadata-first comparator: Kernel English with Hierarchical, Interaction-Amortized Residuals: A Dual-Vocabulary Cognitive Compiler for Efficient Language-Model Reasoning. Corben-authored July 2026 architecture proposal for KERC: protected-object capture, uncertainty-aware normalization, sense-aware Kernel IR, dual surface/core vocabularies, a four-level interaction-amortized residual ledger, exact object storage, grammar-aware macro fusion, structured answer packets, rendering, round-trip verification, versioned migration, and complete rate-compute-fidelity evaluation. Existing chapters are upgraded first; no implementation, benchmark, novelty, efficiency, fidelity, safety, transfer, SOTA, AGI, ASI, or support-state result is inferred.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_airllm_2023	Metadata-first comparator: AirLLM: Scaling Large Language Models on Low-End Commodity Computers. Official implementation comparator for layer-wise model sharding, one-layer accelerator residency, next-layer prefetch, optional storage compression, and original-versus-transformed model storage. Maintainer-reported fit and speed claims are not independently reproduced.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_deepspeed_inference_2022	Metadata-first comparator: DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale. Primary heterogeneous-inference systems source spanning GPU, CPU, and NVMe for dense and sparse Transformer inference. Reported latency, throughput, scale, and model-fit results remain source-scoped and unreproduced.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_flexgen_2023	Metadata-first comparator: FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU. Primary planned-placement source for GPU/CPU/disk tensor storage and access, batching, and optional weight/cache compression under latency-insensitive workloads. Its throughput results are not interactive-latency or local evidence.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_hf_accelerate_big_model_inference_2026	Metadata-first comparator: Hugging Face Accelerate: Loading Big Models into Memory. Official implementation documentation for automatic or explicit GPU/CPU/disk device maps and memory-mapped disk tensors. The documented sequential-dispatch, prefetch, and hard-drive-performance limitations make it a baseline, not a qualification result.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_llama_cpp_memory_mapping_2026	Metadata-first comparator: llama.cpp CLI Memory Mapping, Tensor Placement, and KV Offload Controls. Official consumer-runtime documentation for model load modes, memory mapping, DirectIO, GPU-layer and tensor placement, MoE CPU placement, KV offload, and KV data types. No local model or performance result is implied.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_llm_in_flash_2024	Metadata-first comparator: LLM in a Flash: Efficient Large Language Model Inference with Limited Memory. Primary flash-aware inference source for on-demand parameter loading, I/O cost modeling, transfer reduction, contiguous reads, windowing, and row-column bundling. Sparse/context-adaptive loading is not an exact dense paging result.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_powerinfer_2024	Metadata-first comparator: PowerInfer: Fast Large Language Model Serving with a Consumer-Grade GPU. Primary consumer-inference source for source-reported power-law neuron locality, hot-GPU/cold-CPU placement, adaptive predictors, and sparse operators. Architecture transfer and performance are not locally reproduced.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_vattention_2025	Metadata-first comparator: vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention. Primary counterpoint to non-contiguous PagedAttention layouts: decouples virtual and physical GPU memory while retaining contiguous KV virtual addresses. Reported serving results remain source-scoped.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_infinigen_2024	Metadata-first comparator: InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management. Primary speculative-KV-prefetch source using minimal rehearsal and partial next-layer state to select host-resident KV entries. Prediction, quality, miss, and fallback results are not locally reproduced.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_specache_2025	Metadata-first comparator: SpeCache: Speculative Key-Value Caching for Efficient Generation of LLMs. Primary speculative-KV-prefetch source keeping complete KV state in CPU memory, a low-bit importance copy in VRAM, and predicted next-step KV transfers. Source-reported quality and memory results are unreproduced.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_specoffload_2025	Metadata-first comparator: SpecOffload: Unlocking Latent GPU Capacity for LLM Inference on Resource-Constrained Devices. Primary composition source for target-model offloading, draft-model placement, speculative decoding, and joint tensor/decoding planning. It is not speculative physical-page prediction, and reported results are unreproduced.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_atsinfer_2026	Metadata-first comparator: Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devices. Very recent preprint comparator for tensor-granular static placement, load-aware dynamic transfer, and asynchronous CPU-GPU coordination on consumer devices. Only abstract/metadata were reviewed; reported results are provisional and unreproduced.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_openai_prompt_caching_docs_2026	Metadata-first comparator: Prompt Caching. Current official provider contract for exact-prefix prompt caching, cache-write and cache-read metering, usage receipts, retention, organization isolation, and rate-limit boundaries. Product behavior and prices are time-sensitive; inspected 2026-07-23.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_anthropic_prompt_caching_docs_2026	Metadata-first comparator: Prompt caching. Current official provider contract for reusable prompt prefixes, explicit cache breakpoints, five-minute and one-hour lifetimes, cache creation and read metering, and prewarming. Product behavior and prices are time-sensitive; inspected 2026-07-23.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_gemini_context_caching_docs_2026	Metadata-first comparator: Context caching. Current official provider contract for implicit and explicit context caching, common-prefix placement, cached-token usage reporting, time-to-live, and storage charges. Product behavior and prices are time-sensitive; inspected 2026-07-23.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_vllm_automatic_prefix_caching_2026	Metadata-first comparator: Automatic Prefix Caching. Official vLLM design documentation for block-hash exact-prefix KV reuse, least-recently-used eviction, multi-modal and adapter identity, and tenant cache-salt protection against timing inference. No local serving benchmark was run.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_sglang_radixattention_2024	Metadata-first comparator: SGLang: Efficient Execution of Structured Language Model Programs. Primary RadixAttention and cache-aware scheduling source for structured multi-call language-model programs. Source-reported throughput and theorem scope remain unreproduced.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_prompt_cache_2024	Metadata-first comparator: Prompt Cache: Modular Attention Reuse for Low-Latency Inference. Primary MLSys source for schema-defined reusable prompt modules, positional accuracy, and attention-state reuse across prompts. Source-reported latency remains unreproduced.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_mooncake_2025	Metadata-first comparator: Mooncake: Trading More Storage for Less Computation — A KVCache-centric Architecture for Serving LLM Chatbot. Primary FAST 2025 source for a KV-cache-centric disaggregated serving architecture spanning prefill, decode, DRAM, SSD, and network resources. Production-trace and capacity results remain source-reported.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_cacheblend_2025	Metadata-first comparator: CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion. Primary source for non-prefix and multi-chunk KV reuse with selective recomputation. It makes the cross-attention failure of naïve independent-chunk KV fusion explicit. Source-reported latency and quality remain unreproduced.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
ext_azure_llm_semantic_cache_2026	Metadata-first comparator: Azure API Management LLM semantic cache lookup policy. Official semantic-response-cache policy documentation. It treats vector similarity as an approximate response-reuse decision and warns that a hit can return an incorrect, outdated, or unsafe answer. No local semantic-cache deployment was run.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.
precision_contract	Metadata-first comparator: The Precision Contract: A Functional Rate–Distortion Theory for Behavior-Preserving Neural Computation. Corben-authored July 2026 theoretical and systems paper replacing universal per-weight precision questions with a contract-relative functional rate-distortion problem over complete executable descriptions. It proposes representation canonicalization, protected-behavior contracts, precision fields, progressive base/residual encoding, dynamic routing, full physical and assurance-cost accounting, a Functional Precision Compiler, and scoped precision certificates. Existing chapters are upgraded first; no universal bit bound, implemented compiler, preserved-behavior result, efficiency result, certificate validity, support promotion, SOTA, AGI, or ASI claim is inferred.	No passage-level source claim, local implementation, reproduction, safety, performance, deployment, support-state, or ASI result is established by this reconciliation row.

61.19 Summary

Fast Generation Architectures owns consumer- and request-specific admission of accelerated generation routes plus the complete end-to-end speed receipt. Its thesis is not “make tokens faster.” A route becomes useful only when it improves accepted, verified, delivered work under matched quality, safety, rights, latency, tail, resource, fairness, fallback, recovery, and total-burden gates.

That makes fast generation a bridge between planning, context, routing, runtime, verification, artifact delivery, resource economics, evidence, and inference-acceleration research without absorbing their authority. Planning requests a route, VCM supplies context, Routing exposes qualified candidates, Fast Generation admits the exact mode, Runtime executes it, Verification returns a bounded verdict, Talos and Labor OS deliver artifacts, Benchmaxxing compares complete attempts, and Readiness or Claim Ledgers decide qualification. Speculative, MTP, Medusa/EAGLE-style, lookahead, early-exit, recurrent/state-space, KV-cache, diffusion, and hybrid systems remain comparators or research directions until real matched campaigns earn more.

The bridge is intentionally conservative: speed becomes an architecture win only when the accepted-output ledger survives contact with verification bandwidth, memory pressure, fallback cost, and downstream task success.

The Project Theseus public task-bundle import now adds an implementation-reference counterweight for Fast Generation without promoting speed. theseus_public_task_bundle_import_2026_07_03_local records 64 public BigCodeBench metadata-only tasks, 0 public training rows, 0 task-level regressions, 18 benchmark gates, and visible residual/artifact-gap boundaries from pinned report summaries. Its accepted no-promotion decision evidence_transitions/v1_x_measured/theseus_public_task_bundle_import_no_change.json blocks clean-live-replay, model-quality, benchmark-superiority, generation-speed, useful-solution-per-second, support-state, and chapter-core promotion claims. It is useful to Fast Generation because it preserves task quality, residual, and public-boundary fields beside generation-mode accounting; the clean live Theseus replay remains unclaimed, and it does not prove model quality, benchmark superiority, generation speed, or useful-solution-per-second improvement.

61.20 Evidence reconciliation (2026-07-16)

The invariant protocol, field meanings, and inference limits are stated once in Living Book Methodology. This packet contains only the chapter-specific projection; its authoritative per-atom rows are the fast-generation-architectures slice of experiments/claim_family_terminal_coverage/results/result.json.

The core remains blocked after full attempt at argument support. The strongest family attempt was KERC canonical-language and hierarchical-residual campaign. Its exact boundary is: The historical broad-efficiency transition is N1: the frozen implementation was inadequate, so broader KERC remains untested; two narrow finite observations survive, with no semantic, multilingual, production, energy, or core claim. Across 76 atoms, the terminal ledger records 76 blocked_after_full_attempt.

Chapter-specific field	Value
Family / atom denominator	CF-06 / 76 atoms
Terminal dispositions	76 blocked_after_full_attempt
Core	fast-generation-architectures.core: blocked_after_full_attempt at argument
Core attempted / missing lanes	causal, empirical, executable, formal, source-synthesis / normative, transfer
Attempted local lanes	causal, empirical, executable, formal, source-synthesis
Missing or unproved lanes	normative, transfer
Strongest family bundle	KERC canonical-language and hierarchical-residual campaign (natural_work_and_end_to_end): A 192-record bilingual templated compiler/runtime study with 64 held-out records, five seeds, eight baseline families, 13 ablations, and 20 attacks.
Negative controls	surface and kernel-native baselines; 13 ablations; 20 attacks; ten laundering mutations.
Accepted transitions	v1_0_pilot.fast_generation.no_change
Maximum inference	The historical broad-efficiency transition is N1: the frozen implementation was inadequate, so broader KERC remains untested; two narrow finite observations survive, with no semantic, multilingual, production, energy, or core claim.
Reproduction / next burden	Replay scripts/validate_p4_m8_kerc_campaign.py and scripts/validate_claim_family_terminal_program.py; fill the named atom-specific lanes under a new prospective protocol.

61.21 Handoff

Fast generation reduces time pressure at the output boundary, but the stack must also decide when spending more time is warranted. Governed Deliberation and Test-Time Scaling takes that dual: it records search, revision, verifier scope, budget, stopping, and residual discipline before a slower reasoning result can reach planning. Only after that control plane has produced a bounded candidate does RankFold, NeuralFold, and Artifact Compression ask how reusable artifacts may become smaller without hiding loss, verification, fallback, utility, or cost.