ORBITKVGitHub
Rust 2024 · temporal physical design · NVIDIA H20

Attention semantics.
Compiled into state.

OrbitKV derives when model state can die, synthesizes its physical address plan, and authorizes block reuse only after semantic and GPU execution frontiers agree.

Read the architectureInspect H20 evidence
  • Compiler-generated block plans
  • Proof-carrying reclamation
  • Cost-gated CUDA VMM
orbitkv://layout/full-swa verified
RETENTIONFULL 10 · SWA 52 · W1024 · P16
PAGEDCUDA VMM
analyzelast_read(q, k)01
addressappend + periodic(65)02
viewgeneration_checked03
reclaimdead ∧ done ∧ committed04
fingerprint / sha256frontier / bitemporal
ORBITKV OWNSstate semantics · logical blocks · generations · reclamation proof
ENGINE OWNSmodel execution · scheduling · attention kernels · request protocol

Compilation pipeline

A block manager emitted from semantics.

Ring, append-only log, pinned region, and VMM are lowering choices. The checked lifetime program remains the source of truth.

  1. 01

    Retention IR

    Declare affine may_read(query, key) relations for persistent state.

  2. 02

    Lifetime

    Prove unbounded or fixed-window death from query-key distance constraints.

  3. 03

    Normalize

    Partition state by retirement predicate, not token birth alone.

  4. 04

    Synthesize

    Emit append-only or periodic address programs and minimum slots.

Recorded systems evidence

Capacity becomes admission.

The primary result loads the public gpt-oss-20b checkpoint on one H20. Controlled dummy fixtures remain separate geometry experiments; neither result claims model-quality improvement.

−50.00%

Mistral KV

12K logical prompt with 8K physical slots

1.0028×

Warm runtime

8K PureSWA versus 16K semantic reference

+25.81%

Full capacity

real gpt-oss-20b, same 1.979 GiB KV budget

−20.30%

Owner vs Stock

balanced four-way real-checkpoint ablation

−42.105%

Head-stripe KV

exact multi-scale-window reduction versus max-window allocation

ResultValueContractBoundary
Uniform SWA execution−50.00%Mistral-7B, 12K prompt, 8K physical KV versus 16K referenceone H20; page 1; one request; identical output digest
Real checkpoint capacity+25.81%openai/gpt-oss-20b, Full capacity 47,616 to 59,904same 1.979 GiB KV budget; no radix/spec/overlap/Graph
Real checkpoint makespan−20.30%Owner32 vs Stock128, four balanced execution orders8 x 6000 prompt + 32 decode; identical output-token digests
Capacity prediction4 / 4predicted Full/SWA pools matched fresh SGLang processes exactlygpt-oss-20b, one H20, recorded 1.979 GiB KV budget
Inspect methods and exclusions →

Next gates

Move ownership closer to the GPU.

The next work connects VMM generations to real KV storage and broadens Retention IR without weakening the in-process ownership contract.

NEXT

Graph-stable KV storage

Back real SGLang KV tensors with cost-approved VMM regions.

THEN

Recurrent and prefix state

Compile Mamba state and continuation capsules.

Open source

Compile lifetimes. Reuse with proof.

ArchitectureGitHub ↗