Unfolding the Leech Lattice: Fused Multi-Shell Decoding and VRAM Layouts for 2-Bit LLM Weights

A self-hosted assistant is too expensive because its model uses most of the GPU memory.

What you would change

Try the paper's supported lower-precision weights on a copy of your serving setup.

Try the paper's supported lower-precision weights on a copy of your serving setup.

Compare against

The same model at its original precision, on the same GPU.

1. Locate the paper's implementation and check its license, model support, data needs, and hardware requirements. Stop here if you cannot obtain the required artifacts.
2. Save a small set of representative product tasks and expected outcomes. Keep evaluation tasks out of any training or tuning, and choose quality and cost limits before running.
3. Run the baseline and the proposed change on identical inputs. Keep other settings fixed and repeat paired runs to expose variation.
4. Save per-task outputs, failures, timings, and all usage costs. Review regressions before expanding the test.

What to measure

Peak memory, task pass rate, latency, and total serving cost at the same context and load.

When to keep it

Keep the change only if it meets your preselected quality and latency limits and improves the metric you care about after setup and operating costs. Otherwise retain the baseline. This is an exploration plan, not a validated recommendation.

recommendation

watch

research area

quantization

technical change

Fused multi-shell Leech-lattice 2-bit weight decoding

published

2026-09-02

Existing trellis-based 2-bit kernels such as QTIP dominate this decoder on both traffic and speed on a single commodity GPU, so the reader should use those rather than adopt this serving path.

1. claim 01Paperraft inference
   Read the brief interpretation

   The method supplies the missing multi-shell decoder for Leech-lattice 2-bit quantization, replacing a lookup-table decode with an offline-expanded, warp-divergence-free fused dequantize-plus-matvec kernel at 4.80 bits per weight in VRAM. It costs substantial quality (1.38x perplexity and 14.7 MMLU points at 4B, shrinking with scale) and its in-VRAM rate is far above the on-disk 2-bit rate because the 301-class codebook is too large for a lookup table. It can fail on adoption because the deployed QTIP trellis kernel already beats it 2.27x at near-equal fractions of byte bounds, and on a second memory hierarchy every lattice arm falls below FP16, so the speedup is hardware- and geometry-dependent.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source
2. claim 02Paperraft inference

   Binary bit-plane layout at 4.80 bits per weight runs 2.15x faster than FP16 in decode-phase GEMV; with an int8 output head a served 4B model reaches 87.0 tok/s in 2.60 GB VRAM, but the QTIP trellis kernel reads 2.40x fewer bytes and runs 2.27x faster than the paper's own served layout.

   This is a Paperraft inference. It is not pinned to a verified PDF passage.

   Inspect the source