ForeverBlue816/PrismQuant

PrismQuant

Optimal Null-Space Rotations for Grouped Quantizers

Paper · Hugging Face · Models · Quick start · Method · Figures · Reproduction

Official code and pretrained checkpoints for PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers (arXiv:2609.32429), by Yanlong Chen, Yining Chen, Song Zhang, Amirhossein Habibian, and Yawei Li. See the released models for all Hugging Face checkpoints.

PrismQuant aligns high-energy activation directions with the constant directions of quantization groups. It combines calibrated orthogonal rotations, compact-WY representations and groupwise asymmetric INT4 quantization. This repository contains the core implementation, released model checkpoints and paper figures.

Use Python 3.10+ and a PyTorch installation appropriate for your CUDA version.

git clone --depth 1 https://github.com/ForeverBlue816/PrismQuant.git
cd PrismQuant
pip install -e .
prismquant models

Start with the smallest released model:

from prismquant import load_model

model = load_model("qwen3_0.6b_base")
print(model.generate("The key idea behind quantization is", max_new_tokens=64))

Or use the command line:

prismquant generate --model qwen3_0.6b_base \
  --prompt "The key idea behind quantization is" --max-new-tokens 64

The loader downloads only the selected checkpoint and its required rotation factors, then obtains the matching base model and tokenizer. The default PrismQuant configuration is selected automatically. These are base-model completions, not chat-tuned assistants. See model selection, memory and loading options.

Checkpoint format. The released GPTQ weights are dequantized INT4 values in floating-point tensors. The reference runtime simulates activation and KV quantization while retaining floating-point storage; it is not a packed INT4 serving engine. Use this package's loader, rather than passing the artifact repo to AutoModelForCausalLM.from_pretrained. The separate fused R4 kernels are provided for implementation work.

The model catalog pins the model artifacts and base models to exact revisions. The MoE release includes all expert weights and per-expert rotations. Upstream model licenses apply.

1. Construct: estimate dominant uncentered activation directions and map them to group anchors with Householder reflectors and a balanced permutation.
2. Represent: apply block Hadamard transforms so aligned directions become constant within groups; store the group scale and real affine offset in fp16.
3. Deploy: fold static transforms into weights and use compact-WY factors for the remaining online rotations.

The core rotation can also be used independently of a language model:

import torch
from prismquant import fit_rotation, dynamic_asym_int4

calibration = torch.randn(512, 256)  # replace with your calibration activations
rotation = fit_rotation(calibration, rank=2, group_size=128)
signs = torch.ones(256)
rotated = rotation.apply(calibration, signs)
quantized, scale, offset, codes = dynamic_asym_int4(rotated, 128)
reconstructed = rotation.transpose(quantized, signs)

This small example uses exact second moments; the archived paper pipeline also contains streamed calibration for larger models. See the technical guide for conventions and the kernel guide for the fused path.

See verified evaluation metadata for indexed paper scores, benchmark protocols, and checkpoint links.

The figure gallery includes the author's final Figure 1–6, method overview and activation matrices, with PDF originals and PNG/SVG previews. Figure 5 reports the dedicated deployment implementation; its throughput should not be attributed to the floating-point reference loader above.

prismquant/       Rotation, quantization, GPTQ, folding, loading and CUDA kernels
examples/         Text completion and standalone rotation examples
Figures/          Author-provided paper figures and shareable previews
docs/             Model format, technical guide and reproduction pointers
tests/            Algebra, metadata, loading and generation checks

The complete experiments through E36, per-token results, figure generators and execution records remain in the research archive. The release tree focuses on reusable code and model use.

pip install -e '.[dev]'
pytest -q

See release validation for the test scope and numerical checks.

PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers

Yanlong Chen, Yining Chen, Song Zhang, Amirhossein Habibian, and Yawei Li.

@article{chen2026prismquant,
  title={PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers},
  author={Chen, Yanlong and Chen, Yining and Zhang, Song and Habibian, Amirhossein and Li, Yawei},
  journal={arXiv preprint arXiv:2609.32429},
  year={2026},
  doi={10.48550/arXiv.2609.32429},
  url={https://arxiv.org/abs/2609.32429}
}

Code: Apache-2.0. Model weights retain their respective upstream licenses. Built with Llama for the Llama-derived checkpoints. The GPTQ core is adapted from QuaRot; see third-party notices.