{ "schema_version": 1, "builder_plan": { "scenario": "A self-hosted assistant is too expensive because its model uses most of the GPU memory.", "change": "Try the paper's supported lower-precision weights on a copy of your serving setup.", "baseline": "The same model at its original precision, on the same GPU.", "steps": [ "Locate the paper's implementation and check its license, model support, data needs, and hardware requirements. Stop here if you cannot obtain the required artifacts.", "Save a small set of representative product tasks and expected outcomes. Keep evaluation tasks out of any training or tuning, and choose quality and cost limits before running.", "Run the baseline and the proposed change on identical inputs. Keep other settings fixed and repeat paired runs to expose variation.", "Save per-task outputs, failures, timings, and all usage costs. Review regressions before expanding the test." ], "measure": "Peak memory, task pass rate, latency, and total serving cost at the same context and load.", "decision": "Keep the change only if it meets your preselected quality and latency limits and improves the metric you care about after setup and operating costs. Otherwise retain the baseline. This is an exploration plan, not a validated recommendation." }, "builder_review": null, "arxiv_id": "2609.11716", "title": "Why Does Post-Training Quantization Work?", "summary": "The paper introduces no new technique; it explains why standard post-training quantization, which replaces full-precision weight storage with low-bit weights without retraining, degrades pretrained LLMs far less than naive error-accumulation arguments predict, attributing this to self-cancelling layer errors and LM-head geometry that protects top-ranked tokens. Adoption cost is limited to the usual PTQ workflow (calibration data and a one-time quantization pass via existing tools such as GPTQ or AWQ), with no training infrastructure required. What can fail: the findings are mechanistic and empirical rather than guarantees, so robustness at aggressive bit-widths (e.g., 2-bit), on out-of-distribution tasks, or on architectures not covered by the study must still be validated on the reader's own workload.", "rationale": "The paper provides mechanistic evidence that existing post-training quantization is reliably robust on pretrained models, which justifies adopting standard PTQ for memory-constrained deployment while validating bit-width on the specific workload.", "technique": "Post-training quantization robustness analysis", "family": "quantizacao", "recommendation": "adotar", "published": "2026-09-10", "as_of": "2026-10-18", "editorial_status": "indexed", "source_url": "https://arxiv.org/abs/2609.11716", "independent_implementations": 0, "total_implementations": 0, "stars": 0, "citations": 0, "claims": [ { "claim_id": "claim-01", "statement": "The paper introduces no new technique; it explains why standard post-training quantization, which replaces full-precision weight storage with low-bit weights without retraining, degrades pretrained LLMs far less than naive error-accumulation arguments predict, attributing this to self-cancelling layer errors and LM-head geometry that protects top-ranked tokens. Adoption cost is limited to the usual PTQ workflow (calibration data and a one-time quantization pass via existing tools such as GPTQ or AWQ), with no training infrastructure required. What can fail: the findings are mechanistic and empirical rather than guarantees, so robustness at aggressive bit-widths (e.g., 2-bit), on out-of-distribution tasks, or on architectures not covered by the study must still be validated on the reader's own workload.", "basis": "inferred", "result": "", "baseline": "", "conditions": "", "source_url": "https://arxiv.org/abs/2609.11716", "source_page": null, "source_excerpt": "" } ], "exposure_map": [ { "dimension": "quality", "basis": "not_evaluated", "finding": "", "interpretation": "", "limitation": "", "next_check": "", "source_url": "", "source_page": null, "source_excerpt": "" }, { "dimension": "compute", "basis": "not_evaluated", "finding": "", "interpretation": "", "limitation": "", "next_check": "", "source_url": "", "source_page": null, "source_excerpt": "" }, { "dimension": "latency", "basis": "not_evaluated", "finding": "", "interpretation": "", "limitation": "", "next_check": "", "source_url": "", "source_page": null, "source_excerpt": "" }, { "dimension": "operations", "basis": "not_evaluated", "finding": "", "interpretation": "", "limitation": "", "next_check": "", "source_url": "", "source_page": null, "source_excerpt": "" }, { "dimension": "compatibility", "basis": "not_evaluated", "finding": "", "interpretation": "", "limitation": "", "next_check": "", "source_url": "", "source_page": null, "source_excerpt": "" }, { "dimension": "security", "basis": "not_evaluated", "finding": "", "interpretation": "", "limitation": "", "next_check": "", "source_url": "", "source_page": null, "source_excerpt": "" }, { "dimension": "data_and_training", "basis": "not_evaluated", "finding": "", "interpretation": "", "limitation": "", "next_check": "", "source_url": "", "source_page": null, "source_excerpt": "" }, { "dimension": "reproducibility", "basis": "not_evaluated", "finding": "", "interpretation": "", "limitation": "", "next_check": "", "source_url": "", "source_page": null, "source_excerpt": "" } ], "risks": [], "minimum_test": [], "open_questions": [], "independent_tests": [], "lab_reproductions": [], "report_available": false, "validation_tier": "unknown", "evidence_tier": "unknown", "infrastructure_basis": "unknown", "equations": [ { "anchor": "S3.EGx3-2", "label": "(3)", "section": "3 The Growth of Hidden-Error Norm across Layers", "role": "proposed_method", "latex": "=\\underbrace{\\left(\\frac{\\lVert\\bm{\\mathbf{u}}^{(\\ell)}\\rVert_{2}}{\\lVert\\bm{\\mathbf{h}}^{(\\ell)}\\rVert_{2}}\\right)^{2}\\Bigl[\\left(\\frac{\\lVert\\Delta\\bm{\\mathbf{u}}^{(\\ell)}\\rVert_{2}}{\\lVert\\bm{\\mathbf{u}}^{(\\ell)}\\rVert_{2}}\\right)^{2}-\\bigl(R^{(\\ell-1)}\\bigr)^{2}\\Bigr]}_{\\begin{subarray}{c}\\color[rgb]{0.082,0.3359,0.4961}\\text{relative additional error\n(block-update vs. block-input): }{\\color[rgb]{0.082,0.3359,0.4961}\\bm{T_{\\rm add}}}\\end{subarray}}\\!+\\!\\underbrace{\\frac{2\\langle\\Delta\\bm{\\mathbf{h}}^{(\\ell-1)},\\Delta\\bm{\\mathbf{u}}^{(\\ell)}\\rangle}{\\lVert\\bm{\\mathbf{h}}^{(\\ell)}\\rVert_{2}^{2}}}_{\\begin{subarray}{c}\\color[rgb]{0.7539,0.3672,0}\\text{error interaction: }{\\color[rgb]{0.7539,0.3672,0}\\bm{T_{\\rm inter}}}\\end{subarray}}\\underbrace{-\\frac{2\\langle\\bm{\\mathbf{h}}^{(\\ell-1)},\\bm{\\mathbf{u}}^{(\\ell)}\\rangle}{\\lVert\\bm{\\mathbf{h}}^{(\\ell)}\\rVert_{2}^{2}}\\bigl(R^{(\\ell-1)}\\bigr)^{2}}_{\\begin{subarray}{c}\\color[rgb]{0.1172,0.4297,0.1172}\\text{residual-norm contribution: }{\\color[rgb]{0.1172,0.4297,0.1172}\\bm{T_{\\rm align}}}\\end{subarray}}.", "mathml": "=(∥𝐮(ℓ)∥2∥𝐡(ℓ)∥2)2​[(∥Δ​𝐮(ℓ)∥2∥𝐮(ℓ)∥2)2−(R(ℓ−1))2]⏟relative additional error\n(block-update vs. block-input): 𝑻𝐚𝐝𝐝+2​⟨Δ​𝐡(ℓ−1),Δ​𝐮(ℓ)⟩∥𝐡(ℓ)∥22⏟error interaction: 𝑻𝐢𝐧𝐭𝐞𝐫−2​⟨𝐡(ℓ−1),𝐮(ℓ)⟩∥𝐡(ℓ)∥22​(R(ℓ−1))2⏟residual-norm contribution: 𝑻𝐚𝐥𝐢𝐠𝐧.", "context": "Define R(ℓ):=∥Δ​𝐡(ℓ)∥2∥𝐡(ℓ)∥2. For every ℓ∈{1,…,L} such that 𝐡(ℓ−1),𝐡(ℓ),𝐮(ℓ) are nonzero,", "headline": "", "explanation": "", "objective": "", "symbols": {}, "evidence": "", "evidence_url": "", "source_url": "" }, { "anchor": "S3.E2", "label": "(2)", "section": "3 The Growth of Hidden-Error Norm across Layers", "role": "baseline", "latex": "\\lVert\\Delta\\bm{\\mathbf{h}}^{(\\ell)}\\rVert_{2}^{2}-\\lVert\\Delta\\bm{\\mathbf{h}}^{(\\ell-1)}\\rVert_{2}^{2}={\\lVert\\Delta\\bm{\\mathbf{u}}^{(\\ell)}\\rVert_{2}^{2}}+{2\\langle\\Delta\\bm{\\mathbf{h}}^{(\\ell-1)},\\Delta\\bm{\\mathbf{u}}^{(\\ell)}\\rangle}", "mathml": "∥Δ​𝐡(ℓ)∥22−∥Δ​𝐡(ℓ−1)∥22=∥Δ​𝐮(ℓ)∥22+2​⟨Δ​𝐡(ℓ−1),Δ​𝐮(ℓ)⟩", "context": "For every Transformer block ℓ∈{1,…,L},", "headline": "", "explanation": "", "objective": "", "symbols": {}, "evidence": "", "evidence_url": "", "source_url": "" }, { "anchor": "S4.EGx5-2", "label": "(6)", "section": "4 The Effect of Final Hidden Error on Output Quality", "role": "proposed_method", "latex": "\\frac{\\Delta z_{k}}{z_{k}} =(\\rho\\cos\\alpha-1)+\\rho\\tan\\theta_{k}\\sin\\alpha\\cdot q_{k},\\qquad q_{k}^{2}\\sim\\operatorname{Beta}\\!\\left(\\frac{1}{2},\\frac{d-2}{2}\\right),", "mathml": "Δ​zkzk=(ρcosα−1)+ρtanθksinα⋅qk,qk2∼Beta(12,d−22),", "context": "Conditional on the measured α, assume that the direction of the rotation from 𝐡LM to 𝐡^LM is uniformly distributed over all directions orthogonal to 𝐡LM. For zk≠0 and α,θk∈(0,π), with all expansions taken as α→0+ for fixed d and θk, let Δ​zk:=z^k−zk. Then", "headline": "", "explanation": "", "objective": "", "symbols": {}, "evidence": "", "evidence_url": "", "source_url": "" } ], "equations_status": "selected", "core_kind": "formula", "equations_fetched_at": "2026-09-29", "equations_source_url": "", "equations_source_sha256": "", "exposures_reviewed_at": "", "exposures_source_sha256": "" }