Integration — Loom Kernels

Pick an integration surface

01 / Rust

Contracts and dispatch

Use backend-independent specs and CPU oracles without linking CUDA.

02 / PyTorch

Current-stream operators

Call allocating or caller-owned out variants with capture-safe mutation schemas.

03 / vLLM

Measured opt-ins

Enable only the routes whose exact shapes and semantics passed H20 gates.

04 / Admission

Exact contracts only

Unsupported dtypes, layouts, shapes, or policies never enter Loom.

Build and install the native wheel

python3 -m venv .venv-wheel
.venv-wheel/bin/pip install \
  'setuptools>=80,<82' 'wheel>=0.45' build 'torch>=2.10,<2.12'

CUDA_HOME=/usr/local/cuda-13.1 LOOM_CUDA_ARCHS=90 \
  .venv-wheel/bin/python python/build_wheel.py \
  --cuda-home /usr/local/cuda-13.1 --archs 90 --wheel-dir dist

python3 -m venv .venv-vllm
.venv-vllm/bin/pip install \
  'dist/loom_kernels-1.0.0a1-2cu131torch210sm90-py3-none-linux_x86_64.whl[vllm,test]'

The artifact contains exactly the Rust-ownedlibloom_cuda_bridge.so and boxedlibloom_kernels_torch.so Stable ABI dispatcher. Its manifest binds the Git revision, CUDA toolkit, SM targets, PyTorch range, bridge ABI, and library hashes. Installed packages load only this package-local pair. The current bridge-ABI-2 H20-qualified wheel is not published to a package index yet.

Call PyTorch directly

from loom_kernels import (
    greedy_sample_logprobs,
    greedy_speculative_verify,
    min_p_filter_,
    selected_token_logprobs,
    silu_and_mul_dynamic_fp8,
)

fp8_output, block_scales = silu_and_mul_dynamic_fp8(
    gate_and_up_bf16,
    group_size=128,
)

token_ids, logprobs, ranks = greedy_sample_logprobs(logits)
verified_ids, accepted, emitted = greedy_speculative_verify(
    draft_ids_i32, target_argmax_ids_i64, bonus_ids_i32,
    cumulative_draft_lengths_i32, max_draft_tokens,
)
logprobs, ranks = selected_token_logprobs(logits, sampled_ids_i64)
min_p_filter_(sampling_logits_f32, min_p_f32)

vLLM 0.24/0.25 opt-ins

Route	Enable
Add+RMSNorm IR	`ir_op_priority={"fused_add_rms_norm": ["loom_cuda"]}`
SiLU-and-Mul	`LOOM_KERNELS_ENABLE_SILU_AND_MUL=1`
SiLU-and-Mul→block FP8	`LOOM_KERNELS_ENABLE_SILU_AND_MUL_FP8=1`
RoPE+paged-KV	`configure_vllm_rope_paged_kv(...)`
Short paged decode	`LOOM_KERNELS_ENABLE_PAGED_DECODE_ATTENTION=1`
Greedy sampled logprob	`register_vllm_greedy_sample_logprobs()`
Greedy speculative verify	`register_vllm_greedy_speculative_verify()`
Selected-token logprob	`register_vllm_selected_token_logprobs()`
Min-P	`LOOM_KERNELS_ENABLE_MIN_P=1`

Admission happens before Loom dispatch.

Loom does not copy, cast, reshape, or alter sampling policy to force eligibility. The adapter calls Loom only for an exact contract; otherwise vLLM keeps ownership of the operation.

Compatibility is explicit

Fresh installs of the current bridge-ABI-2 wheel with official vLLM 0.24.0 and 0.25.1 packages each pass the 225-test H20 suite on PyTorch 2.11. The same exact wheel passes 138 applicable Loom gates on PyTorch 2.10 without recompilation. It includes speculative verification and static FP8 KV quantize-on-write but has not been published. A pinned Qwen2.5 draft/target engine gate proves exact native/Loom speculative invocation on vLLM 0.24; vLLM 0.25 has source-suite compatibility evidence but no transferred model-level result. Existing model-level performance artifacts remain tied to the version recorded in each result; compatibility does not transfer a speedup claim. Read the compatibility matrix.

Read the exact contracts

Route-specific shapes, dtypes, environment variables, build details, and validation commands live in the complete vLLM provider guide. The evidence index records accepted, parity, fallback, and rejected experiments.