ScaleSearch: Stop Letting the Absolute Maximum Monopolize FP4 Scaling

0. Link verification, versions, and resource audit

The links below were visited on 2026-09-11 actually accessed. Conference identity prioritizes official MLSys/PMLR proceedings; arXiv dates follow submission history. “Not public/not found” means no verifiable entry was found through the official paper page, arXiv Code/Data/Media, GitHub title search, Hugging Face Papers, Papers with Code, Semantic Scholar, or project searches. Core numbers come from the 16-page official MLSys PDF, not web abstracts.

Version inconsistency alert. The MLSys PDF lists the author name as Shirley Wu, while the official webpage and arXiv use Xiaoxia Wu. The arXiv HTML still contains “Anonymous Authors” and “Under review by MLSys” template text, whereas the final PDF names authors. This report follows the official proceedings identity but does not assume the two English names reflect one person's renaming without an author explanation. Another detail: the proceedings index says Proceedings of Machine Learning and Systems 8while the PDF footer says “9th MLSys Conference”; one is a volume number, the other the conference edition, so they do not conflict.

1. Candidate pool, 14-day exclusions, and random selection

First check ml-report-index.md: the past 14 days already covered QServe(a W4A8KV4 serving system) and Lumos(training-performance modeling), so those papers were excluded and highly overlapping mixed-precision serving kernels/training predictors avoided. An initial pool of 27 papers from MLSys 2025 was built, and the prescribed draw shuf -i 1-27 -n 1 selected #23 APOLLO. It was then discovered that MLSys 2026 proceedings were already available with 135 accepted papers. Keeping a 2025-only pool would violate the preference for formal papers from the past 12 months, so that draw was invalidated—not because the paper was unpopular or its results unattractive.

The rebuilt pool primarily used MLSys 2026 proceedings, with arXiv, OpenReview, Hugging Face Daily Papers, Papers with Code, Semantic Scholar, and ACL Anthology title searches checking identities/resources. A systems topic does not require adding ACL papers merely to diversify sources; those auxiliary searches found no extra core-paper code or data. This produced 30eligible candidates, followed by shuf -i 1-30 -n 1 and selected #24 ScaleSearch. Total redraws: 1, solely because newer official proceedings were discovered.

2. Motivation: the 4-bit bottleneck is not only too few bits, but a poor scale

NVFP4 represents continuous values using one E2M1 four-bit float per element plus one shared E4M3 eight-bit scale per 16 elements. E2M1 has only

ℛ_E2M1 = {0,  ± 0.5,  ± 1,  ± 1.5,  ± 2,  ± 3,  ± 4,  ± 6}.

so each original value can ultimately lie only on sℛ_E2M1 . Standard scaling uses the block's absolute maximum x_max :s = round_UE4M3(x_max/6), roughly mapping the largest magnitude to 6. This heuristic serves one extreme rather than minimizing error across all 16 values. Core §§3–4 (pp.4–5) note that the E4M3 scale has three mantissa bits, so Blackwell directly supports several neighboring non-power-of-two scales. That existing grid permits searching for a lower-MSE point without changing the storage format.

The hardware motivation is direct. The introduction (p.1) cites Blackwell peak NVFP4/MXFP4 matmul throughput relative to FP8: 2×on B200 and 3×on B300. NVIDIA's Table 1 and Memory section explain that sharing an E4M3 scale across 16 values averages about 4.5 bit/value, with model memory about 1/3.5of FP16 and 1/1.8of FP8. These are format/hardware limits, not ScaleSearch's own end-to-end gains. Its narrower question is whether the same 4.5-bit representation and Tensor Core instructions can make fewer rounding errors.

Figure 3 and the text expose an internal inconsistency worth auditing. The y-axis of Figure 3 (PDF p.5) clearly shows MSE falling from approximately 0.0090 to 0.0066, a relative decrease of (0.0090−0.0066)/0.0090 = 26.67%, broadly consistent with 27% in the abstract and 26% in the contributions list. Yet the text on p.6 gives a starting value of 0.0990while still claiming about 25%. Recomputing from those textual values would give 93.33%. This report therefore adopts the plotted 0.0090 as credible and treats 0.0990 as a tenfold decimal typo, rather than combining mutually inconsistent numbers.

Downstream pain is uneven. PTQ Table 1 (p.9) shows ordinary NVFP4 reducing DeepSeek-R1-Distill-Qwen-1.5B MATH-500 from 64.6 to 51.6, a loss of 13.0 pp; Qwen3-8B GPQA falls from 51.3 to 42.4, losing 8.9 pp. Conversely, quantized Qwen3-8B MATH-500 at 73.1 slightly exceeds full precision at 72.8, showing that benchmark/sampling noise intertwines with quantization error. ScaleSearch is most valuable where max-based scaling is clearly mismatched and low-bit errors affect task performance, not necessarily for every tensor.

3. Paper narrative: from local enumeration to complete FP4 attention

The paper has two nested narratives.

The first is numerical. Starting from the standard scale's E4M3 bit pattern, the authors reinterpret it as int8, enumerate offsets f, requantize 16 values for each candidate, and select the lowest MSE. Exhaustive enumeration of [−127,127] can find the globally optimal representation but costs too much. Figure 4 shows bimodal offsets near 0 and 4–5 for both Gaussian data and real Llama 3.1 8B Key states, motivating the fixed range [−2,6] of nine candidates. The second mode has a discrete-geometric explanation: default scaling encodes the maximum as 6; enlarging the scale by roughly 6/4 = 1.5 encodes it as 4 while potentially aligning the other 15 values better with the grid (pp.5–7).

The second narrative is systems integration. Search is applied to ModelOpt weight PTQ, SageAttention3 video-diffusion attention, and a new ScaleSearchAttention (SSA). SSA also adds Q/K Hadamard incoherence processing, an SVD-derived magnitude-balancing transform, a sink-aware KV cache retaining high-precision first/latest blocks, and an NVFP4 Tensor Core path through QKᵀ, softmax probabilities P, V, and PV. SSA results in Tables 3/4 therefore cannot be attributed entirely to ScaleSearch; component ablations in Table 5 provide the attribution evidence.

The assumptions fall into three levels:

1. Local-surrogate assumption: lower per-block MSE improves model function. Often reasonable, but not guaranteed because attention and residual paths differ in sensitivity to error directions.
2. Distribution-transfer assumption: Gaussian blocks and real Key states share bimodal offsets, so a fixed [−2,6] range transfers to weights, Q/K/V/P, LLMs, and video diffusion. Only one real Key-tensor histogram is shown (Figure 4, p.5), without model/layer coverage checks.
3. System-amortization assumption: scale search can be fused or amortized in offline PTQ, so 1.74× local quantization cost does not destroy end-to-end throughput. Video latency supports roughly 2%–3% extra time, but language models have only attention-kernel TOPS curves, without TTFT, TPOT, or throughput/latency SLOs.

These levels express the actual tradeoffs: lower MSE costs more candidates; more stable attention costs high-precision boundary caches and offline matrix transforms; stronger hardware efficiency requires Blackwell-only deployment.

4. Formalization and procedure

4.1 NVFP4 representation and standard quantization

versus x ∈ ℝ¹⁶, the NVFP4 representable set is

𝒱_NVFP4 = {sq ∣ q ∈ ℛ_E2M1¹⁶, s ∈ ℛ_UE4M3}.

A N × D tensor is represented as (Q,S), where Q ∈ ℛ_E2M1^N × D、S ∈ ℛ_UE4M3^{N × (D/16)}and decoded as

X̂ = decode(Q) ⊙ Broadcast(S).

These follow core §3 (p.4). Four-bit values plus one eight-bit scale per 16 values average 4 + 8/16 = 4.5 bits/value, excluding the per-tensor FP32 global scale.

4.2 Discrete optimization over nearby representable scales

Let the standard scale be s⁽⁰⁾. Interpret its UE4M3 bit pattern as an integer and add offsets:

s^(f) = reinterpret(reinterpret(s⁽⁰⁾,int8)+f,UE4M3).

ScaleSearch solves

$$f^*=\arg\min_{f\in[f_{\min},f_{\max}]}\sum_{i=1}^{16}\left(x_i-s^{(f)}\,\mathrm{round}_{E2M1}(x_i/s^{(f)})\right)^2.$$

With the default [−2,6] , only nine neighboring scales are evaluated per block. For block length B = 16and candidate count K = 9, naive search complexity is O(KB), with unchanged storage format. Extra cost occurs during quantization, not matmul. Algorithm 1 (p.5) also checks 1 ≤ scale_int8 + f ≤ 127to remain within the positive UE4M3 scale range.

Input: 16-value block x, offset range [-2, 6]
1  xmax = max |x_i|; s0 = round_UE4M3(xmax / 6)
2  best_loss = +∞
3  For f = -2 ... 6:
4      sf = E4M3 bit pattern of s0 plus f
5      qf = round_E2M1(x / sf)
6      loss = Σ_i (x_i - sf*qf_i)^2
7      If loss improves, save sf, qf, f
8  Return optimal block scale and 16 FP4 values

4.3 Magnitude balancing that preserves attention scores

SSA first uses a Hadamard transform to reduce outliers, then introduces invertible R：

Q′ = QR^−T,   K′ = KR,

Thus

Q′K′^T = QR^−TR^TK^T = QK^T.

Appendix A (p.16) sets X = 𝔼[Q^TQ]、Y = 𝔼[K^TK]and minimizes

min_Rtr(R⁻¹XR^−T) ⋅ tr(R^TYR).

If X^1/2Y^1/2 = USV^T. The paper gives R = Y^−1/2VS^1/2, making both traces equal to tr(S), with minimum product tr(S)². This preserves scores and balances second-order energy, but lower energy implying lower quantization error remains a surrogate relationship. The appendix gives no final task-error bound for discrete E2M1 rounding.

4.4 Mixed-precision KV and FP4 kernel path

Figure 8 (p.8) splits both QKᵀ and PV into high/low-precision subblocks. The first block and latest incomplete block remain full precision; completed interior blocks become NVFP4. Once new tokens fill the latest block to B , it enters the 4.5-bit KV cache. High-precision storage occupies only O(B)and does not grow with context; however, B ≥ 16because V uses groups of 16 along the token dimension. Q/K/V/P all undergo per-row scaling followed by ScaleSearch. NVFP4 MMA uses FP32 accumulators, so the authors claim no explicit dequantization in the matmul path.

5. Key mechanisms and engineering details

Core §5.3 text/tables do not identify the GPU, nor does Figure 9's caption. Since SA3 targets Blackwell, one cannot simply label these results B200, B300, or RTX 5090. SageAttention3 itself reports 1038 TOPS on RTX 5090 (pp.1,7), but that does not establish ScaleSearch used the same device.

6. Datasets, benchmarks, and metrics

The paper claims five general-capability benchmarks, but Table 1 actually has four columns: GPQA, MATH-500, AIME-120, and MMLU (§5.1, p.9). This is a text/table counting inconsistency, not a transcription omission. AIME-120's composition by year/question is also unclear, affecting reproducibility.

7. Complete main-results matrices and recomputation

7.1 Weight PTQ: full Table 1 transcription

From core Table 1 (p.9). Parentheses give reported five-run dispersion, but the caption does not define standard deviation versus standard error. MMLU has one run and no parentheses. Bold marks each model's best.

ScaleSearch versus ordinary NVFP4 is recomputed below. Absolute changes use percentage points (pp); relative change is (ScaleSearch-NVFP4)/NVFP4：

Thus “across all benchmarks” is not strictly true: among eight model×metric cells, 7/8 improve, while Qwen3-8B AIME-120 falls 0.7 pp. Being within one standard deviation is a reasonable qualitative explanation, but without paired tests, CIs, or the five raw runs, “not significantly worse” cannot be rewritten as equivalent. More unusually, 8B MATH-500 rises from 72.8 at full precision to 88.1 with ScaleSearch,15.3 pp higher. This likely includes decoding/evaluation randomness, but no configuration is available to verify it.

7.2 Video diffusion: full Table 2 transcription

From core Table 2 (p.10). Relative to SA3, Mochi VQA-a/t/FScore changes are +7.789 pp（+15.69%）、+14.624 pp（+26.24%）、+0.269（+14.35%）; for CogVideoX, +2.838 pp（+3.89%）、+0.961 pp（+1.28%）、+0.284（+5.55%）. However, CLIPSIM changes in the two models are −0.0003（−0.16%）、−0.0012（−0.65%）, so “matching” means nearly unchanged, not nondecreasing on every metric. Without standard deviations, 0.0001-level CLIP-T differences should especially not decide winners.

7.3 WikiText-2 PPL: six precision paths

From core Table 3 (p.10). SSA's PPL reductions versus Naive-FP4 are, respectively, 0.5011（8.35%）、0.7652（22.51%）、0.3170（2.75%）、0.1411（1.67%）. The “up to about 22%” claim holds for Llama 70B, but SSA remains 0.0794 (+3.11%) above full precision. Qwen3-8B differs by only 0.0005 (+0.006%), closest to near-zero loss. Relative gains range from 1.67% to 22.51% across four models, so larger scale does not guarantee more benefit: Llama and Qwen differ substantially even at 8B.

The paper cannot supply the requested unified five-benchmark×six-method matrix: PTQ has three methods×four columns, PPL six methods×four models, and video three methods×five metrics. Combining accuracy, PPL, and video scores would create false comparisons. This report therefore preserves the complete original tables without inventing a fifth benchmark.

7.4 GPQA Diamond and system costs

Core Table 4 (p.10), on Llama 3.1 8B Instruct, reports Full Prec 31.81、SA3 26.26、SSA 32.32. SSA versus SA3 is +6.06 pp（+23.08%）, not the introduction's generic “+5 points.” It also exceeds full precision by 0.51 pp, but superiority cannot be established without variance.

Core Tables 6–7 (p.11) report costs:

The narrower [-1,1] search takes 0.0328 ms, or 1.27×. Its MSE, PPL, and task accuracy are unreported, preventing a quality-per-extra-0.01-ms Pareto curve. Calling 1.74× minimal overhead is justified only for offline PTQ or when large matmuls amortize it end to end. Per-token dynamic quantization requires kernel-fusion and batch/sequence-distribution evidence.

8. Ablations, sensitivity, and transitions

Core Table 5 (p.10) ablates SSA only on Llama 3.1 8B PPL:

This table shifts the paper's center of gravity:ScaleSearch is an elegant, general, format-preserving local improvement, but nearly lossless FP4 attention mainly depends on high-precision boundary KV and transforms. No full factorial ablation isolates mixed precision alone, IP alone, or magnitude-transform removal while retaining Hadamard, so their interactions remain unknown.

Two sensitivity analyses provide plots without raw tables. Figure 3 (p.5) shows MSE dropping rapidly with candidate count and plateauing near ten candidates; the default nine lies near the elbow. Figure 7 (p.6) shows the MSE gap with/without ScaleSearch shrinking as block size grows through 1, 2, 4, 8, 16, and 32, explaining why 16-value NVFP4 suits search better than 32-value MXFP4. Figure 5 (pp.6–7) reports MSE reductions around 27% for NVFP4, 11% for MXFP6 E2M3, and 8% for MXFP4, with some hypothetical formats approaching 80%. These format-level transitions suggest finer scale mantissas and smaller blocks make neighborhood search more valuable.

The paperNot reported: layer/head sensitivity;[-1,1] with [-2,6] quality comparisons; real-model results beyond B=16; calibration-set size; magnitude-transform regularization; number of high-precision boundary blocks; and end-to-end latency percentiles across batch/sequence settings. The default lies near the synthetic-MSE elbow, but is not established as the globally optimal system configuration for all models.

9. Statistical significance and evidence strength

Only PTQ Table 1 explicitly averages, except MMLU, over 5 runs, but its parenthesized statistic is undefined. Explaining the 8B AIME −0.7 pp as within one standard deviation suggests that interpretation without formally specifying it in the caption. There are no paired seeds, t-tests, bootstrap CIs, or multiple-comparison corrections. MMLU is single-run; video Table 2, PPL Table 3, GPQA Table 4, ablation Table 5, kernel Table 6, and latency Table 7 all lack standard deviations.

Evidence therefore falls into three tiers:

• Stronger: synthetic MSE within the same format and an explicit discrete objective. Because the candidate set includes the default scale, per-block MSE on the evaluated samples cannot exceed the default.
• Moderate: seven of eight PTQ cells and PPL on all four models improve; direction is consistent, but randomness is inadequately described.
• Weaker: video quality/latency on 12 prompts, one GPQA setting, single-curve 98.3%/97.5% points, and extending nearly lossless claims to production serving.

Candidate-set monotonicity must be distinguished from model-quality monotonicity: ScaleSearch's chosen MSE on the same block cannot exceed f = 0, yet task accuracy can decline, as Qwen3-8B AIME demonstrates. Minimizing local L2 error does not guarantee monotonic improvement in discrete problem-solving outputs.

10. Contributions, reusable designs, and transfer potential

The first genuine contribution formulates BFP scale calibration ashardware-compatible discrete neighborhood optimizationrather than training a new quantizer. It preserves the checkpoint's 4.5-bit layout, adds no inference-time codebook, includes the default scale, and integrates readily into PTQ. For any microscaling format with mantissa scales, define neighborhoods by bit patterns, then narrow them using real-tensor offset histograms.

The second contribution shows that mapping the maximum to the largest codeword is not always optimal. Offsets 4–5 encode the maximum as 4 rather than 6, sacrificing one extreme's codeword utilization for better grid alignment of the remaining values. This transfers to nonuniform codebooks, per-group integer quantization, and KV caches: candidates may vary zero-points, clipping ratios, or a few codebook temperatures rather than scales alone.

The third contribution assembles SSA: FP4 QKᵀ/PV, 4.5-bit KV, FP32 accumulation, high-precision boundary blocks, and magnitude balancing form an executable path. The reusable lesson is:theoretical quantization-kernel throughput requires outlier transforms, protection of sensitive tokens, and cache layout to become practical performance. This matters more than any single PPL value.

Transfer should use task-sensitive error objectives: Hessian-weighted block loss for weight PTQ, QKᵀ/PV output errors for attention, and noise-stage/layer-specific ranges for generative models. Online search becomes more expensive; a compromise is learning per-layer offset priors offline and evaluating only two or three likely candidates.

11. Technical lineage: how problems propagate and where the direction changes

11.1 NVFP4: providing a searchable scale grid

Problem solved. NVIDIA's NVFP4 explanation combines E2M1 values, 16-value microblocks, E4M3 scales, and per-tensor FP32 scales to balance 4-bit compute and accuracy on Blackwell. Table 1 explicitly contrasts MXFP4's 32-value, power-of-two scales.
Remaining gap. The format permits fine-grained E4M3 scales but retains the single-point amax/6 heuristic, leaving neighboring mantissa points unused.
Core response. ScaleSearch turns existing but unused scale resolution into a candidate space without changing the format.
Design transfer. Small 16-value blocks and non-power-of-two E4M3 steps directly enable offset search. With MXFP4 E8M0, Figure 6 shows mostly offsets 0/1 and smaller gains.

11.2 QuIP#: from weight incoherence reduction to Q/K outlier reduction

Problem solved. QuIP# uses randomized Hadamard transforms to improve incoherence, followed by E8 lattice codebooks and fine-tuning for ≤4-bit weight PTQ. The final ICML paper's Figure 2/Algorithm 1 (pp.2–3) specifies the pipeline; Tables 5–6 (pp.10–11) show generation throughput with quantized weights.
Remaining gap. It primarily quantizes weights. Hadamard transforms neither select NVFP4 block scales nor handle causal KV/P.
Core response. SSA transfers Hadamard IP to Q/K, then adds scale search and a QKᵀ-preserving R transform.
Design transfer. The inherited principle is to make distributions more uniform before low-bit discretization. Table 5 attributes about 6.5× more PPL benefit to IP+magnitude transforms than to scale search, so prior work deserves appropriate credit.

11.3 StreamingLLM: from retaining attention sinks to high-precision boundary KV

Problem solved. StreamingLLM finds that retaining only a recent window causes PPL to explode beyond cache size, while retaining four initial tokens restores stability. Figure 3, Tables 1–2 (pp.4–5), and Figure 10 (p.11) report scaling to 4M tokens and up to 22.2× speedup over recomputation.
Remaining gap. It addresses token eviction/streaming, not interior-KV numerical precision, and has no NVFP4 Tensor Core path.
Core response. SSA retains interior tokens but quantizes their blocks to NVFP4, keeping the first/latest blocks high precision.
Design transfer. The insight that initial tokens are structurally sensitive becomes a mixed-precision policy. However, SSA does not report exact retained-token counts, model-specific sink widths, or long-context needle/reasoning results.

11.4 SageAttention3: from the first FP4 attention kernel to better scales and causal caches

Problem solved. SageAttention3 uses 1×16 microscaling FP4 for QKᵀ/PV and two-level P quantization. Table 1 (p.6) reports NVFP4 CosSim 99.52%; p.1 reports RTX 5090 throughput of 1038 TOPS, about 5× its fastest FlashAttention baseline.
Remaining gap. The core paper calls SA3's public algorithm numerically unstable in causal settings (ScaleSearch §5.2, p.9), and SA3 is not a complete 4.5-bit causal KV-cache solution.
Core response. ScaleSearch is added directly to SA3 for video. Language attention is reorganized as SSA with IP, magnitude balancing, and high-precision boundary caches.
Design transfer. SA3's 1×16 blocks, FP4 P path, and hardware optimizations provide the system foundation. Video results are SA3 plus a better scale chooser; SSA results combine several components anew.

11.5 Nonlinear technical evolution tree

                         ┌─ QuIP / QuIP#: Hadamard incoherence ─┐
Traditional BFP: shared exponent ─┤                            ├─→ Q/K distribution shaping
                         └─ MX: 32 values + power-of-two scale ─┐
                                                               ├─→ NVFP4: 16 values + E4M3 scale
NVIDIA Blackwell Tensor Core ───────────────────────────────────┘             │
                                                                             ├─→ ScaleSearch: discrete neighborhood MSE search
FlashAttention: online softmax / IO-aware ─→ SageAttention3: FP4 QKᵀ + PV ──┤
                                                                             │
StreamingLLM: initial tokens are sinks ─→ high-precision boundary blocks ─────┤
                                                                             ↓
                SVD magnitude balancing + ScaleSearch + FP4 MMA + 4.5-bit KV = SSA
                                      │
                                      ├─ mixed-precision KV preserves most accuracy
                                      └─ scale search supplies small, stable local corrections

Evolution is driven first byhardware-format changes: scales gain mantissas rather than exponents alone, enabling fine-grid search; then bybottleneck migration: faster FP4 matmul makes quantization, softmax, and KV layout system bottlenecks; finally bychanges in error location: static weight-PTQ errors become dynamic Q/K/P/V errors in causal attention, requiring mixed precision and distribution transforms. The turning point is not another bit removed, but a format that makes scale selection an optimization variable.

11.6 Cross-paper comparison

12. Stated limitations and boundary conditions

The paper has no separate Limitations or Broader Impact section, so two or three author-stated limitations cannot be invented. Three explicit boundaries can be extracted from the text:

1. Format boundary. The authors mainly implement NVFP4 because it is, to their knowledge, the only format with floating-point scales and modern hardware acceleration (§4, p.5). Other formats appear only in Figure 5 simulations. Architecture-agnostic describes the algorithm, not existing cross-hardware kernels.
2. Causal-baseline boundary. The authors explicitly call public SA3 code numerically unstable in causal settings, so Table 3 implements SA3's algorithm in their FP4 simulator (§5.2, p.9). Language quality therefore compares simulated numerical paths, not two public kernels end to end on hardware.
3. Mixed-precision boundary. The mixed-precision cache requires B ≥ 16and permanently retains O(B) full-precision boundary storage (§4.2, p.8). “All-FP4 attention/KV” is therefore not literal: most blocks are FP4, sensitive blocks full precision.

The authors do not discuss missing code, the 12-prompt sample, statistical significance, or conflicting main-text numbers. Those are independent audit findings, not stated limitations.

13. Independent critique: eight evidence-to-inference chains

1. [Attribution] Because removing ScaleSearch in Table 5 changes PPL only to 5.4977→5.5024（+0.086%）, while removing mixed-precision KV changes it to 5.5768（+1.439%）, nearly lossless SSA cannot primarily be credited to its named component. Contributions should separate local scale optimization from the integrated attention system.
2. [Internal consistency] Because p.6 states 0.0990→0.0066 ≈25%while Figure 3 actually shows about 0.0090→0.0066 =26.67%, at least one tenfold decimal error exists. The figure supports the abstract's 27%, but the textual numbers cannot be cited directly.
3. [Statistics] Because only PTQ reports five runs, parentheses are undefined, other main tables lack variance, and quantized 8B MATH exceeds full precision by 15.3 pp, evaluation randomness may change several conclusions. Without raw seeds and unified decoding, task performance cannot be claimed to track MSE monotonically.
4. [Baseline fairness] Because the authors simulate SA3 for language PPL due to unstable public causal code, and SSA includes extra high-precision caches and SVD transforms, Table 3 is not a head-to-head changing only scales in one hardware kernel. It establishes a better numerical recipe than a simulated baseline, not a superior production kernel.
5. [System extrapolation] Because 98.3%/97.5% comes only from 32K attention TOPS, language models lack TTFT, TPOT, batch sweeps, concurrent requests, or P99, and video uses only 12 prompts, negligible system overhead applies only to tested kernels/generation scripts, not online-serving SLOs.
6. [Distribution coverage] Because [-2,6] is selected from standard Gaussian data and one Llama 3.1 8B Key-state histogram yet applied to weights, Q/K/V/P, four LLMs, and two video models. Evidence for a universal fixed range is insufficient; if P or V offsets lie outside those modes, nine candidates may not be the best budget.
7. [Reproducibility] Because core code, ModelOpt patches, the PyTorch simulator, GPU model, calibration examples, and full hyperparameters are unpublished, readers cannot verify 0.0449 ms, Table 3, or Figure 9. Public SageAttention/ModelOpt reproduce source components, not the paper's method.
8. [Objective mismatch] Because the algorithm minimizes unweighted per-block MSE yet AIME shows −0.7 ppand CLIPSIM declines for both models, local Euclidean error is not sufficient for task risk. The next step should compare Hessian/attention-output-weighted search rather than merely enlarge the same MSE candidate set.

14. Open challenges: actionable head-to-head experiments

14.1 Theory

• MSE versus functional error. Fix one NVFP4 checkpoint and nine candidates; compare four choosers: block MSE, Hessian-weighted MSE,∥QK^T − Q̂K̂^T∥_F² with ∥PV − P̂V̂∥_F² . Evaluate WikiText-2, LongBench, GPQA, and AIME with identical seeds. Stronger correlation between weighted objectives and task gains would test critique eight.
• Search-range optimality. Record exhaustive [-127,127] oracle offsets for each layer/head, then compare [-1,1]、[-2,6], a learned top-three prior, and adaptive early stopping. Plot oracle coverage, quantization time, and PPL together. The current paper gives costs without quality curves.
• Error-propagation bounds. Within one attention block, quantize Q, K, V, P separately and jointly; estimate Lipschitz bounds from local MSE to output error/softmax KL. Separate sink-heavy and uniform heads to identify conditions under which lower block MSE improves stability.

14.2 Engineering and applications

• Real Blackwell end-to-end evaluation. Using public commits on B200/B300/RTX 5090, compare BF16 FlashAttention-4, SA3, SA3+ScaleSearch, and SSA. Control batch and prompt/output lengths; report TTFT, TPOT, tokens/s, P50/P99, HBM, and power. Current evidence covers only TOPS and two video-script timings.
• Memory-matched KV comparison. At equal average bits per token, compare SSA, KIVI, KVQuant, Kitty, and FP8 on Llama 3.1/Qwen3 over 4K–128K contexts with needle, LongBench, reasoning, and PPL tests. SSA's 4.5-bit interior plus full-precision boundaries cannot be directly compared with nominal 4-bit/2-bit methods.
• Offline versus online quantization. Measure search costs separately for one-time weight PTQ, prefill K/V writes, and single-token decode writes; compare LUT/learned-offset selection. A 0.0449-ms 2048² Gaussian matrix does not resolve small-batch decode launch/branch costs.

14.3 Emerging directions

• Cross-format search. On the same Blackwell/AMD platforms, compare NVFP4 E4M3, MXFP4 E8M0, MXFP6 E2M3, and FP8 while controlling total bits and hardware peaks. Use identical oracle/neighborhood search to separate mantissa-scale benefits from small-block benefits. Figure 5 is only software simulation.
• Generation-stage adaptation. Dynamically choose search ranges/precision by video-diffusion timestep or LLM prefill/decode phase and token uncertainty. Compare with fixed [-2,6] at equal average quantization FLOPs and measure the quality–latency Pareto frontier. Ambient/staged-precision ideas may suit this better than fixed search throughout.
• Safety boundaries for learned choosers. Predict offsets with a small classifier while retaining f = 0 and a local MSE verifier. Compare errors, fallback rate, and P99 with exhaustive nine-way search. The aim is to reduce 1.74× cost without losing the noninferiority guarantee from including the default candidate.

15. Recent-work radar

The following official MLSys 2026 paper pages were accessed; they are not sources for core-paper numbers.

1. FlashInfer-Bench: connects agent-generated kernels, real traces, correctness/performance evaluation, and deployment replacement into a loop. It supplies the public kernel-benchmark/reproducibility framework missing from ScaleSearch.
2. FlashAttention-4: Blackwell asynchronous MMA, softmax rescaling, and TMA pipelines, reaching 1613 TFLOPs/s BF16 on B200. This is a strong full-precision baseline SSA must test on the same hardware.
3. Kitty: dynamic channel-wise precision boosts for nearly 2-bit KV, reporting seven tasks and 2.1×–4.1× throughput. It directly exposes a missing comparison with SSA's token-block mixed precision.
4. MixLLM: global mixed precision across output features, with two-step dequantization/kernel pipelines realizing efficiency. Bit allocation remains a major optimization axis beyond scale search.
5. SkipKV: jointly suppresses redundant long-CoT generation and KV storage at sentence level, with up to 1.7× throughput. Reducing tokens may outperform saving a few more bits per token; compare at equal quality and answer length.
6. Demystifying the MoE Serving Tax: finds FLOP-equivalent MoE can be 2–3× slower than dense models, with load imbalance affecting prefill/decode oppositely. System evaluation should decompose workload phases, not stop at one kernel.
7. MoEBlaze: addresses the MoE memory wall through token dispatch, buffer materialization, and checkpoint co-design, claiming use in Meta recommendation production. Its real deployment context is valuable relative to ScaleSearch, though workload and baseline fairness still require reading the full paper.

16. Conclusion: reuse scale search, but do not overattribute nearly lossless attention

ScaleSearch's insight is simple: NVFP4 E4M3 offers finer scales than E8M0, so amax/6 need not be the sole answer. Choose among nine neighboring representable scales by MSE for each 16-value block; preserve the format and default candidate while integrating easily into offline PTQ. Its strongest quantitative evidence is PTQ 7/8 cells outperforming ordinary NVFP4, Qwen3-8B MATH-500 +15.0 pp, and approximately 26.67% synthetic MSE reduction in Figure 3.

Nearly lossless ScaleSearchAttention is an integrated-system result. Llama 70B PPL improving 3.4000→2.6348 is striking, but removing ScaleSearch worsens PPL only 0.0047 versus 0.0791 for high-precision boundary KV. With unpublished implementation, simulated causal SA3, no real language-serving latency, and the paper's 0.0990 decimal/five-benchmark counting errors, the most defensible conclusion is:scale-neighborhood search is a low-risk component worth transferring; SSA is a promising prototype still lacking a complete chain of reproducible hardware evidence.

Evidence and production notes

• The official MLSys PDF was read page by page, pp.1–16. Main results come from Tables 1–7, Figures 3–9, Algorithm 1, and Appendix A, cited in the relevant sections.
• Related works read page by page: SageAttention3 (especially pp.1–8), QuIP# (pp.1–11), and StreamingLLM (pp.1–12); NVIDIA's NVFP4 technical explanation was read in full.
• Relative percentages and pp changes were recomputed from paper tables to two decimal places; unreadable plot points were not invented as tabulated values.
• Missing statistics, unpublished code/data, and unspecified hardware are marked explicitly; web abstracts were not substituted for paper numbers.
• HTML passed the Warm Editorial structural validator (0 warnings), with static responsive-CSS checks for light/dark themes, 640-px mobile, and print rules. The runtime lacks a /usr/bin/chromium、chromium-browser or /opt/pw-browsers executable, so the installed Playwright module could not launch a browser. Screenshot-based visual QA is therefore not claimed. Tables use horizontally scrollable containers, SVGs use responsive viewBox, and theme/text colors come from dual-theme variables.
• ml-report-index.md This issue's entry has been appended.