Loom KernelsRust × CUDAGitHub

Documentation

Start with the boundary.

Loom is a Rust-first operator backend—not an inference engine or tensor framework. Pick the contract you need, validate it on CPU, then qualify CUDA and the engine path independently.

Choose a path

01 / Understand

Read the contracts

Architecture, supported operators, admission gates, and deliberate exclusions.

Open the design →
02 / Integrate

Connect an engine

Install the native wheel, preserve current streams, and enable narrow vLLM routes.

Open integration →
03 / Evaluate

Inspect the evidence

Separate operator, CUDA Graph, engine, parity, fallback, and rejected results.

Open the evidence index →
04 / Build

Pick the next boundary

Follow the measured roadmap instead of implementing an unbounded operator checklist.

Open the roadmap →
05 / Compatibility

Check the qualified matrix

Separate source support, framework versions, H20 validation, and binary portability.

Open compatibility →
06 / Contribute

Propose a measured operator

Name the engine call site, contract, baseline, and exit gates before a large implementation.

Open contributing →

Validate the Rust core

The default workspace has no CUDA requirement. It checks dtype, shape, buffer, aliasing, and deterministic reference behavior first.

git clone https://github.com/feichai0017/loom-kernels.git
cd loom-kernels

cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace --all-targets
cargo check --workspace --release
Support is a ladder.

Contract correctness, accelerator correctness, named-baseline speed, framework dispatch, engine invocation, and serving benefit are six separate gates. Passing one never implies the next.

Workspace layers

LayerResponsibility
loom-kernelsPublic contracts, capabilities, and deterministic CPU references.
loom-cudaSafe stream/buffer ownership, checked dispatch, and benchmarks.
loom-cuda-sysInternal CUDA launch ABI, compilation plumbing, and handwritten kernels.
loom-cuda-bridgeThe only framework boundary into safe borrowed Rust dispatch.
pythonCurrent-stream PyTorch operators and narrow vLLM integration.

Qualify CUDA explicitly

CUDA_HOME=/usr/local/cuda-13.1 \
LOOM_CUDA_ARCHS=90 \
cargo bench -p loom-cuda --features cuda \
  --bench add_rms_norm -- \
  --dtype bf16 --rows 8 --hidden-size 4096

Benchmarks check their GPU result against the CPU oracle before reporting CUDA-event latency. The complete result archive lives in the H20 evidence index.