OrbitKV / Documentation
A cache beyond
GPU memory.
OrbitKV stores reusable KV in DRAM and SSD so repeated requests can reuse work after GPU eviction. Start with installation, connect your engine, then use metrics and benchmarks to tune capacity for your workload.
- Install a wheel and start the Cache Manager.
- Enable the adapter in your vLLM or SGLang environment.
- Verify an external hit and inspect cache capacity.
Start with vLLM →
Install the connector, start the manager, and verify an external cache hit.
Start with SGLang →
Enable the GPU-page linker and reuse cached prefixes through the same manager.
The pinned official engines are vLLM 0.31.0 and SGLang 0.5.21. Start with dense single-rank recovery. Public lifecycle, broader topology, native P/D faults and HA retain separate gates. Check compatibility and support.
Getting started
Install, select a supported profile, and verify an external hit.
Deployment
Connect engines, share capacity, and select a topology.
Operations
Configure capacity, inspect metrics, and diagnose failures.
Architecture
Understand identity, ownership, and storage paths.
Benchmarks
Read scoped results and reproduce matched measurements.
Development
Contribute against the single execution queue.
Build and contribute
Technical pages are published directly from the repository's documentation. Start with the contributor guide, test gates, and benchmark harness.