Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models

0. Link verification and material completeness

All URLs below were actually visited. If a site provides only an entry point without a separate resource, it is marked not released/not found; search snippets are not treated as paper evidence.

Index-site verification notes. Candidate search covers accessible official OpenReview/ICLR pages, arXiv, ACL Anthology, PMLR, Papers with Code, and Hugging Face Daily Papers. No stable direct Semantic Scholar page was found for the core paper, so third-party summaries supply no numbers. All quantitative claims were checked against tables, figures, or text in paper PDFs/HTML.

1. Candidate pool, deduplication, and random-draw audit

1.1 Deduplication over 14 days

First read ml-report-index.md. Recent multimodal coverage includes MM-RLHF(preference alignment/safety), Catch Your Emotion(emotion-aware MLLMs), and BridgeVLA(robot vision–language–action). These themes are excluded as core topics. Neighboring research may appear in the pool, but the drawn core must remain distinct. The final topic is internal VLM spatial–temporal binding, without repeating recent papers or main questions.

1.2 Candidate pool(26 papers)

Priority goes to vision/multimodal papers from the past12 months confirmed accepted at ICLR2026 or formally published at top2025 conferences. Order was fixed before drawing. “Official list entry” means title/acceptance verified in the official ICLR2026 OpenReview list without a stable individual direct ID.

1. MMR-V: What’s Left Unsaid?
2. Vision Language Models are Biased
3. PostAlign: Multimodal Grounding as a Corrective Lens for MLLMs
4. Towards Text-Mask Consistency in Medical Image Segmentation
5. MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs
6. SportR: A Benchmark for MLLMs on Sports Understanding
7. SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning?
8. Linear Mechanisms for Spatiotemporal Reasoning in VLMs
9. Memento: An All-Day Proactive Assistant for Ultra-long Video
10. Towards Native Vision-Language Primitives at Scale
11. MANZANO: A Simple and Scalable Unified Multimodal Model
12. Better Together: Leveraging Unpaired Multimodal Data
13. Thought, Less Accuracy? The Dual Nature of Reasoning in VLMs
14. A Latent Partial Causal Model for Multimodal Learning
15. Long-tailed Test-Time Adaptation for Vision-Language Models
16. MRMR: Multidisciplinary Multimodal Retrieval Benchmark
17. Rethinking Bottlenecks in Safety Fine-Tuning of VLMs
18. SPIKE-RL: Video-LLMs Meet Bayesian Surprise
19. To Sink or Not to Sink: Visual Information Pathways in Large VLMs
20. Fine-R1: Make MLLMs Excel in Fine-Grained Visual Recognition by CoT Reasoning（Official ICLR2026 list entry）
21. V2P-Bench: Evaluating Video-Language Understanding with Visual Prompts（Official list entry）
22. AGILE: Agentic Jigsaw Interaction Learning
23. Towards Faithful Reasoning in Remote Sensing（Official list entry）
24. Efficient Discriminative Joint Encoders for Large-Scale Vision-Language Reranking（Official list entry）
25. Thyme: Think Beyond Images（Official list entry）
26. OmniMouse: Scaling Properties of Multimodal, Multitask Brain Models

1.3 Random procedure

After fixing the pool, actually executed:

$ shuf -i 1-26 -n 1
8

Paper8 was selected. Its complete26-page version, acceptance page, open code, core tables, and appendix ablations support an in-depth reading, sono redraw was needed. Popularity, citations, or subjective preference did not replace the draw.

2. Motivation: where do VLMs lose spatial relations, beyond accuracy alone?

Classic spatial evaluations reveal symptoms without locating their source. Kamath et al. test18 vision–language models on controlled positions; VQAv2-tuned BLIP achieves only 56%versus humans at 99%(EMNLP2023 abstract/main text pp.1–2). The 43 pp gap does not necessarily mean the model cannot see: it may detect both objects but fail to bind directions; the encoder may encode positions that cross-modal projection loses; or middle layers may be correct but language output misreads them.

The core paper separates three falsifiable questions:

1. Representation existence. Can object-token hidden states linearly decode position when the same object appears in different grid cells?
2. Causality. Does replacing a left object’s spatial representation with a right object’s flip the answer as predicted, rather than merely perturb it meaninglessly?
3. Error localization. When the answer is wrong, is the error already in the visual encoder or introduced later during image–text binding?

Linear separability does not imply use, and correlation does not imply causation. The paper combines probing, mirror patching, arbitrary steering, symmetric visual/text diagnostics, and auxiliary training losses. Its main quantitative motivation is causal: left/right spatial-ID steering produces median binary-belief swaps of 64.6%versus random-noise control 29.5%(main paper p.5, approximately lines 295–307), with a reported mean increment over chance of 43.6%. Thus, decoded directions participate in final judgments, beyond classification.

A caution: the caption/abstract vicinity states 64.4%while main results state 64.6%there exists 0.2 pp , an internal inconsistency. This report primarily follows64.6% in the main results while retaining the discrepancy. It does not alter the qualitative conclusion but cautions against reproducing numbers from the abstract alone.

3. Argument, core assumptions, and trade-offs

3.1 Narrative

The paper transfers entity binding from language models to multimodality: Feng & Steinhardt found additive, interchangeable binding IDs linking entities and attributes. The core paper hypothesizes analogous VLM spatial-slot IDs linking object content with position. It controls objects/positions in synthetic4×4 grids, trains linear directions, then patches across layers and modalities.

If correct, the hypothesis predicts:

• Position can be read from object tokens through a low-rank linear subspace;
• left/right and up/down directions have approximately consistent geometry;
• transferring an ID between objects changes answers about that object;
• mirror patching correct image states and incorrect text states localizes errors to encoding or binding;
• adding ID structure as a training loss improves spatial accuracy faster than language modeling alone on the same data.

Experiments broadly support this chain with declining strength: representation existence and simple binary interventions are strongest; training gains use one small LLaVA setup; temporal mechanisms rely mainly on qualitative curves; depth relations become confused. The appropriate conclusion is manipulable linear binding for simple spatial–temporal relations, not a unified explanation of VLM spatial reasoning.

3.2 Assumptions and costs

4. Equation-level formalization

4.1 Hidden states and belief changes

For an image with objects(o_a,o_b), let the object-related hidden state at layer(ell) be(h_o^{(\ell)}). For binary answers(y^+,y^-), define the logit belief difference:

[ B(x)=z(y^+\mid x)-z(y^-\mid x). ]

After intervention(I) on hidden states, the paper uses

[ \Delta B = B(I(x))-B(x) ]

to measure steering rather than only final binary flips, preserving movements in the correct direction that do not cross the decision boundary(main Eq.1, p.3).

4.2 Constructing spatial IDs

For position(p=(i,j)) and object(o), collect object-token representations(h_{o,p}^{(\ell)}). Average across objects to remove object semantics:

[ s_{p}^{(\ell)} = \mathbb{E}{o}[h{o,p}^{(\ell)}] - \mathbb{E}{o,p'}[h{o,p'}^{(\ell)}]. ]

The left/right direction is the difference between two position-group means:

[ d_{LR}^{(\ell)} = \mathbb{E}{p\in L}[s_p^{(\ell)}]-\mathbb{E}{p\in R}[s_p^{(\ell)}], ]

The up/down direction(d_UD^{(\ell)}) is analogous. Ridge regression also maps position encoding(psi(i,j)) to spatial IDs:

[ \hat{s}_{i,j}^{(\ell)} = W^{(\ell)}\psi(i,j)+b, ]

Low-rank constraints(\operatorname{rank}(W)\le r) are examined. At(r=3), all three models have(R^2) at least 0.854(appendix Table 2, around p.20; see Section 7’s matrix), supporting the specific claim that 2D position largely occupies a very low-dimensional subspace.

4.3 Mechanistic approximation

The theoretical sketch approximates object-position representations through attention-value projections:

[ s_{i,j}\approx W_O W_V P\left(\psi(i,j)-\bar{\psi}\right), ]

Here(P) is cross-modal projection and(W_V,W_O) are attention value/output projections. Final relation logits are also approximated by linear readout:

[ z_r \approx w_r^\top (s_{i,j}-s_{i',j'})+c_r. ]

These expressions explain how positional encoding→low-rank spatial ID→relation logit could form a linear chain. They are a mechanistic sketch, not an equality proof for full softmax attention, MLPs, and residual networks; global model linearity cannot be claimed.

4.4 Mirror patching and arbitrary steering

Mirror patching constructs samples differing only in reversed relations, such as A left of B versus A right of B. The source object’s spatial component is injected into the target:

[ \tilde h_o^{(\ell)} = h_o^{(\ell)} + \alpha\big(s_{p_{src}}^{(\ell)}-s_{p_{tgt}}^{(\ell)}\big), ]

Default steering scale is(\alpha=5)(appendix p.16). Arbitrary steering directly rewrites the target-position ID without paired mirror images. Outputs tracking the target position provide stronger evidence than linear classification alone.

4.5 Spatial training loss

The authors align object-token spatial IDs with target IDs alongside language loss:

[ \mathcal{L}=\mathcal{L}{LM}+\lambda\mathcal{L}{spatial},\qquad \mathcal{L}{spatial}=\sum_o|g(h_o^{(\ell)})-s{p(o)}|_2^2. ]

No complete(\lambda) sensitivity curve is reported. The selected configuration improves accuracy at 3,200 steps from 85% to 91%, without establishing stable gains for arbitrary weights.

4.6 Reproducible pseudocode

Inputs: object set O,4×4 positions P, relational prompt templates Q, VLM M
1  Generate controlled images across object pairs, sizes, and positions
2  Extract object-token hidden states h[o,p,ℓ] at each layerℓ
3  Center across objects to obtain spatial IDs s[p,ℓ]
4  Train linear/low-rank mappings ψ(i,j) -> s[p,ℓ], recording R²
5  Construct mirrored relation pairs; patch object, text, and image tokens separately
6  Compute ΔB and compare with equal-norm random noise
7  Symmetrically patch correct/incorrect examples to localize encoder/binding errors
8  Fine-tune with L_spatial against LM-only controls using equal data and steps
9  Repeat for video: forward/reverse,8-frame synthetic trajectories, MVBench Scene_QA
Outputs: linear readability, causal effects, error hierarchy, training gains

5. Key mechanisms and engineering details

6. Datasets, benchmarks, and metrics

6.1 Synthetic4×4 grid

The main diagnostic set places objects in discrete cells, varying left/right, up/down, front/back/size relations. Counterfactual pairs can differ only in position, ideal for causal patching, but the distribution is unusually clean. Appendix A.2(around pp.12–13) specifies baseline coverage of 90 object pairs ×4 sizes, totaling 86,400 images on a canvas of 896×896; resources include 55 rendered Objaverse objects. The set supports linear-readability, ID sample-efficiency, prompt-robustness, size-ablation, and arbitrary-steering tests.

6.2 COCO-Spatial / WhatsUp family

Real-image spatial QA tests transfer of synthetic directions to natural images. Accuracy is primary; training records steps, LM loss, spatial-ID loss, and COCO validation accuracy. WhatsUp’s historical controlled benchmark gives humans99% versus BLIP56%, but different training setups prevent treating that gap as the core paper’s direct improvement.

6.3 MVBench Scene_QA and reversed videos

The authors reverse frames to observe internal representations and answers about event order/temporal content. Synthetic videos use 8 framesand 5 object pairs covering 61 temporal arrangements. MVBench Scene_QA provides natural-video QA. Metrics remain accuracy/belief difference, but no reconstructable five-benchmark×six-method video matrix is published, so missing entries are not fabricated from plots.

6.4 Metric interpretation

• Accuracy： Final-answer accuracy: easy to read but misses changes near decision boundaries.
• Belief shift (\Delta B)： Intervention-induced change in correct-versus-incorrect answer logits, measuring directional causal effects.
• Variance explained / (R^2)： Fraction of spatial-ID variance explained by a low-rank position mapping. High(R²) supports low-dimensional structure, not a direct guarantee of task accuracy.
• Mann–Whitney U： Compares intervention-effect distributions on correct/incorrect samples. Significance trends are reported without complete exact p-values, effect sizes, or multiple-comparison corrections.

7. Main-results matrices

7.1 Evidence-coverage matrix

The paper lacks a complete five-benchmark×six-method table under one protocol. Rather than fabricate a matrix across incompatible tasks/models/metrics, the table lists verifiable experimental units, retaining unreported values as missing instead of estimating plots.

7.2 Counterfactual cross-table of object tokens and relational directions

From core Appendix Table 1. “ID-LR / apply-obj” identifies IDs from left/right directions but intervenes on object tokens, testing whether direction discovery and application must occur at the same location.

For all four models, extracting IDs from object positions and applying them to object tokens is best within each row. Qwen-3B’s81.23 exceeds48.46 from applying the same ID to LR tokens by 32.77 pp / 67.62%; Qwen-7B gains 24.39 pp / 50.85%. Spatial information is therefore object-bound rather than only an abstract relation-word direction. Values should retain the original table’s intervention-effect/success interpretation, not be relabeled accuracy.

7.3 Low-rank mapping from positional encoding to spatial IDs

The rank 2→3 jump is large, especially LLaVA’s0.396, but does not imply that space is intrinsically three-dimensional. Under the chosen encoding/centering, a third linear degree of freedom may absorb grid/global bias or another positional variation. Full rank 1–8 spectra, singular values, and cross-dataset repeats are needed before calling it a phase transition.

7.4 Spatial auxiliary-loss training curves

Both groups have identical accuracy for the first 1,600 steps. Differences emerge at 2,400 and reach6 pp at 3,200. Meanwhile both LM losses are already below 0.01, while spatial-ID loss falls from 0.41 to 0.33. This supports language objectives saturating before spatial constraints finish shaping representations. Both groups eventually saturate/overfit around 90%, however, suggesting improvedsample/optimization efficiencyrather than a higher unlimited-training ceiling.

8. Layerwise ablations, sensitivity, and possible phase transitions

8.1 Layer depth

Effective steering concentrates in middle layers, scanned from 9 to roughly two-thirds depth. Early layers lack stable visual–language object binding; late layers have consolidated answer directions. This echoes NAACL 2025 information-flow findings of early visual convergence and late task differentiation, but no unified normalized-depth table establishes an exact cross-architecture critical point.

More importantly, appendix confidence intervals usethe three best steering layersrather than preregistered layers or all-layer means. Selection inflates effects, so 64.6% is an attainable middle-layer effect, not expected performance at a random layer.

8.2 Sample size

The text says as few as 9 object pairsexplain at least 50% of spatial-ID variance; the caption lists ablations using 6/20/56 object-pair images. These may count samples differently or reflect version inconsistency. Direction discovery appears data-efficient, but the exact turning point at 6 versus 9 is uncertain. Missing pointwise means±standard deviations and random-subset stability limit confidence in “9 pairs suffice.”

8.3 Object size and prompts

Replacing baseline sizes with 80/128/176-pixel objects reduces explained variance by about 10–20%, but it remains at least 50%; prompt wording gives similar results(appendix pp.18–19). This rules out memorizing one size/template, but not occlusion, perspective, or long-tail categories, since resizing still occurs on regular canvases.

8.4 Directions and depth

Left/right and up/down are relatively clear, but above/front and behind/below are confused(main p.6). This counterevidence suggests3D depth words may map to 2D projections or corpus co-occurrence. A unified spatial ID should preserve depth independence under controlled camera changes; current evidence supports strong2D layout but weak depth.

8.5 steering scale

Default(α=5) lacks a dose–response curve from 0 upward or hidden-state-norm-normalized scales. Small natural-direction effects cannot be distinguished from large OOD shocks or model-specific scales. No verifiable transition threshold exists here; threshold claims require reproduction.

8.6 Time dimension

Eight-frame synthetic videos,61 arrangements, and reversed MVBench videos suggest temporal IDs, but lack frame-count sweeps(2/4/8/16/32), speed variation, and long-video delay tests. Long-term memory effectiveness and direct transfer of spatial low-rank conclusions to time remain unsupported.

9. Statistical significance and evidence strength

Mann–Whitney U tests compare correct/incorrect or intervention/control distributions, with graphical confidence intervals. The main text lacks systematic exact p-values, U statistics, effect sizes, sample sizes, seed counts, and multiple-comparison corrections. Training tables also lack means±standard deviations. Therefore:

• Directional effects are clear in the authors’ samples, especially64.6% versus 29.5%, a35.1 pp gap;
• stable intervals across seeds, prompts, and object samples are not established;
• small0.2–2 pp differences cannot be rigorously compared;
• best-three-layer selection means ordinary CIs do not automatically cover selection uncertainty;
• the+6 pp training gain may include initialization/batch-order effects without multiple seeds.

An ideal reproduction stratifies by model×object subset×prompt, preregisters layer ranges, reports bootstrap95% CIs, Cliff’s delta or rank-biserial effects, applies Benjamini–Hochberg correction, and nests layer selection within validation.

10. Related work: problem and technical lineage

10.1 Kamath et al.2023: image understanding without reliable spatial judgments

Problem solved. Controlled COCO-prep, GQA-prep, and RealCLEVR sets show18 VL models far below humans on simple spatial relations: BLIP-VQAv2 gets56% versus 99% human performance.
Remaining gap. Evaluation identifies failures but not whether they originate in perception, cross-modal binding, or language output.
How the core paper responds. Counterfactual positions of the same object, object-token IDs, and mirror patching decompose failure by module/layer.
How the key design transfers. Controlled minimal pairs transfer from benchmark design into causal intervention examples; COCO-Spatial bridges training and natural-image validation.

10.2 Feng & Steinhardt2024: binding IDs from text to visual objects

Problem solved. Language models bind entities and attributes through additive IDs. Control accuracy is >99%, unilateral entity/attribute intervention yields <3%, and bilateral intervention restores >97%(pp.4–5). Directions transfer across tasks: Table 2 gives capitals0.88, parallel0.87, shapes0.71, fruits0.80, bios0.47, versus zeros/random around 0.30/0.31.
Remaining gap. Text entity–attribute binding need not equal visual object–position binding; visual encoders/projectors introduce bottlenecks.
How the core paper responds. Average across objects to extract spatial IDs, patch image/object-word tokens, and use symmetric visual/text interventions to localize errors.
How the key design transfers. Interchangeable IDs plus consistent bilateral substitution directly form the causal-test framework.

10.3 Hernandez et al.2024: linear relation readout is not universally valid

Problem solved. Some relations admit a single approximate linear transform from subject to object representations; the project also notes that accurate behavior can coexist with nonlinear internal relation encoding.
Remaining gap. Focuses on textual factual relations, not object positions/video order or binding directions to visual objects.
How the core paper responds. Moves linear relation readout into multimodal middle layers and uses object-level steering to test output use.
How the key design transfers. Retains linear maps, low-rank analysis, and causal replacement. The warning that nonlinear representations can be correct means failed probes do not prove absent information.

10.4 Zhang et al.2025: information flow identifies layers, not variables

Problem solved. Tracks visual contributions across LVLM reasoning tasks, finding early convergence, late task differentiation, and task-dependent cliff layers(NAACL 2025 Long, pp.2289–2299).
Remaining gap. Aggregate contributions cannot identify object, position, or answer-prior content; correlation curves do not prove causality.
How the core paper responds. Uses spatial IDs as named variables for precise object/text/image-position patching.
How the key design transfers. Retains layer scans but moves from information quantity to binding type and intervention-induced answers.

10.5 Chen et al.2025 / AdaptVis: complementary attention-focus and ID repairs

Problem solved. Analyzes spatial failures through attention, finding target-region attention correlated with success. AdaptVis delivers training-free zero-shot gains on WhatsUp/VSR of up to approximately 50 absolute percentage points(ICML 2025 abstract/main results).
Remaining gap. Looking at the right region can still bind objects incorrectly; salient attention need not carry causality.
How the core paper responds. Directly manipulates object-position IDs to distinguish visual encoding from later binding errors.
How the key design transfers. The methods can combine: AdaptVis repairs where to look, then spatial-ID loss repairs object–position binding, creating a testable sequential intervention.

10.6 Nonlinear technical evolution

Controlled spatial evaluation(WhatsUp,2023)
├── Symptoms: accuracy collapse ──> Attention-focus diagnosis/AdaptVis(2025)
│                         └── Repairs where to look, not necessarily binding
├── Mechanisms: textual binding IDs(ICLR2024)
│       ├── Additive, interchangeable entity binding
│       └──> Visual object–spatial IDs(core paper,2026)
│              ├── Low-rank position mapping
│              ├── Object/text/image mirror patching
│              └── Spatial auxiliary loss
├── Representations: linear relation decoding(ICLR2024)
│       └──> Low-rank readout + causal steering
└── Layers: cross-layer visual information flow(NAACL 2025)
        └──> Middle-layer window + error localization

Unresolved branch:2D left/right/up/down ──X──> 3D depth, occlusion, long sequences, open-vocabulary reference

10.7 Cross-paper comparison

11. Genuine contributions, reusable design, and transfer potential

11.1 Actual contributions

1. Turns spatial failures into manipulable variables. Compared with accuracy or attention maps alone, spatial IDs specify object ownership, directions, and replacement operations.
2. Connects representation, causality, diagnosis, and training. Linear readout shows existence; steering shows use; mirror patching localizes failure; auxiliary loss tests repair. Few mechanism papers cover all four.
3. Cross-architecture low-rank evidence. Rank3 achieves(R²=0.854–0.903) on LLaVA, LLaMA3.2-VL, and Qwen2.5-VL, so controlled2D structure is not unique to one model.
4. Distinguishable error types. LLaVA errors more often involve visual encoding/object detection, while LLaMA errors appear later in cross-modal binding(main pp.6–7). This informs targeted repair rather than indiscriminate data expansion.

11.2 Reusable designs

• Causal minimal-pair generator: Reusable for tasks comparing the same objects with only relations changed.
• Object-centered IDs: Cross-object averaging removes content semantics, potentially extending to color, counting, order, and role identity.
• Bilateral mirror patching: Routes multimodal diagnosis between encoder and projector/LLM failures.
• Low-rank spectra as mechanism compression: Tests for compact intermediate attribute representations without training a large model.
• Mechanism-guided auxiliary loss: When LM loss saturates while structural loss declines, it can inform stopping or curricula.

11.3 Transfer potential

Direct applications include binding document fields to 2D layouts, lesions to quadrants/organs in medical imaging, and video entities to event order. Robotics could bind visual-object IDs to action coordinates, but BridgeVLA and related work expose viewpoint changes, control latency, and long-horizon planning. Simple4×4 IDs can represent local state, not replace a world model.

12. Limitations acknowledged by the paper

1. Simple tasks. The authors acknowledge mostly simple spatial relations and appearance/order-focused temporal tasks, excluding compositional reasoning(Limitations, pp.8–9).
2. Limited model scale. Compute limits models to about 14B; equivalence in larger open or frontier closed models is unknown(Limitations, pp.8–9).
3. Narrow demonstrated uses. Broader diagnostics/training remain future work, without production or safety-critical validation(Limitations, pp.8–9).

13. Independent critique(evidence→inference)

1. [Statistics/selection bias] Because CIs use the best three steering layers without preregistration or nested-selection correction,64.6% resembles an attainable peak rather than an unbiased population effect; cross-model averages may be inflated.
2. [Internal consistency] Because captions/abstracts say64.4% while main results say64.6%, and sample counts differ between6 and at least 9 pairs, code reproduction takes precedence over secondary copying. Comparisons precise to 0.1 pp are unreliable.
3. [Intervention validity] Because default scale5 lacks dose–response curves or hidden-norm normalization, flips may partly reflect OOD activations. Small-scale monotonic effects, equal-norm random controls, and activation-norm matching are needed for robust causal interpretation.
4. [External validity] Because images mainly use 896×896 regular grids, fixed sizes, and separated objects, while natural scenes include occlusion, perspective, and semantic co-occurrence,(R²≥0.854) cannot directly generalize. The 10–20% variance-explanation drop after resizing already signals domain sensitivity.
5. [3D semantics] Because above/front and behind/below are confused, spatial IDs may primarily encode2D projection/corpus relations rather than coordinate-independent3D geometry. Counterfactual camera poses are needed to distinguish them.
6. [Training attribution] Because spatial-loss and control groups match through1,600 steps and differ by 6 pp only at 3,200, without multiple seeds, optimization timing or regularization remains a possible explanation. Gains are not yet uniquely attributable to ID geometry.
7. [Weak temporal evidence] Because video coverage systematically describes only 8 frames,5 pairs, and 61 arrangements without complete per-model tables, temporal evidence is weaker than spatial evidence. It is a proof of concept, not long-video memory validation.
8. [Scale and closed models] Because experiments stop around 14B and exclude frontier closed models, whether low-rank IDs persist, split, or become nonlinear with scale remains open.

14. Open challenges: actionable head-to-head experiments

14.1 Theory

T1: linear IDs or local tangent approximations? Compare rank 3 linear maps, two-layer MLPs, kernel ridge, and local linear models on identical models/data with matched parameters/validation. On natural-image counterfactuals report(R²), steering AUC, and OOD position generalization. If MLP improves probing but not steering, linear mechanisms gain support; otherwise revise the linearity claim.

T2: meaning of the third dimension. Measure rank 1–8 singular spectra and cross-seed CIs, predicting IDs separately from 2D coordinates, homogeneous coordinates((x,y,1)), and depth(z). Compare explained variance and transferable steering to determine whether rank 3 captures bias or geometric depth.

T3: attention versus residual paths. At one layer, separately path-patch(W_V/W_O), MLPs, and residual streams, comparing belief shifts with whole-token patching. If only whole-token patching works, the proposed linear attention chain is insufficient.

14.2 Engineering and applications

E1: AdaptVis × Spatial Loss2×2 factorial experiment. On WhatsUp, VSR, and COCO-Spatial compare baseline, AdaptVis, spatial loss, and both at equal model/training budgets. Report accuracy, target-box attention IoU, ID(R²), and inference latency to separate looking from binding.

E2: natural-domain robustness. Create fixed-relation groups with no occlusion, partial occlusion, camera rotation, and conflicting background semantics. Compare original models, ID steering, LoRA, and full fine-tuning with at least 5 seeds/bootstrap CIs, measuring relative-position accuracy and unrelated-task degradation.

E3: serving utility. Use error localization as a runtime gate: recrop/redetect encoder errors, rewrite prompts or correct activations for binding errors. Compare with self-consistency and tool-based detectors on accuracy, extra FLOPs, P95 latency, and false-rejection rates.

14.3 Emerging directions

N1: long-video temporal IDs. At 8/32/128/512 frames compare fixed IDs, RoPE temporal readout, external memory, and streaming VLMs. Use event-reversal minimal pairs to report temporal steering, late-sequence degradation, and memory. Stable512-frame performance is needed for long-term binding claims.

N2:3D/embodied coordinate systems. Change cameras in one robot scene while preserving world-coordinate relations. Compare image-centric IDs, camera-centric3D tokens, and world-centric scene graphs on relation invariance, grasp success, and cross-camera transfer, testing whether 2D IDs can become action-space variables.

N3: multi-object composition and reference. Scale from 2 to 4/8/16 objects with references such as the book behind the second red cup on the left. Compare single low-rank IDs, slot-based binding, and explicit scene graphs; report error growth with object count, not just average accuracy.

15. Recent work to follow

1. PostAlign, ICLR 2026： Uses multimodal grounding to correct bias, complementing spatial-ID intermediate-representation repair: one changes output alignment, the other object–position binding.
2. MetaSpatial, ICLR 2026： Targets metaverse/3D spatial reasoning, naturally testing whether 2D low-rank mechanisms extend to camera/world coordinates.
3. SpatiaLab, ICLR 2026： Provides broader spatial evaluation to avoid closed-loop validation only on custom4×4 grids.
4. To Sink or Not to Sink, ICLR 2026： Tracks visual channels in large VLMs, complementing named spatial-ID variables: where information goes versus which variables are bound.
5. SPIKE-RL, ICLR 2026： Uses Bayesian surprise to improve Video-LLMs, enabling comparison of active keyframe selection with continuous temporal binding.
6. MMR-V, ICLR 2026： Multi-frame evidence reasoning requires implicit information composition, exposing cross-frame relations absent from 8-frame synthetic ordering tasks.
7. SpatialVLM, CVPR 2024： Trains spatial abilities with about 10M images and 2B VQA examples, representing the data-scaling route for equal-budget comparison with mechanism-guided loss.
8. AdaptVis, ICML 2025： Up to about 50 pp spatial gains are striking, but object-binding diagnostics are needed to determine whether gains reflect correct attention or stronger prompt priors.

16. Final assessment

The paper offers a reusable mechanistic-research template:Controlled-variable generation → low-rank readout → causal substitution → error localization → mechanism-guided training. Its strongest findings are 64.6% median belief swaps under spatial-ID steering versus 29.5% random noise, and rank 3 mappings explaining85.4%–90.3% variance across three architectures. Compact positional labels for object binding become a testable hypothesis.

Conclusions should remain at middle-layer linear mechanisms for simple 2D relations, not a unified spatiotemporal theory. Two numerical inconsistencies, selected-layer bias, absent scale5 sensitivity, single-seed+6 pp training evidence, incomplete temporal matrices, and confused depth semantics remain. Preregistered head-to-head causal tests on natural images, camera changes, long videos, and object compositions are more valuable than additional probes alone.

17. Sources and verifiable references

• Core paper HTML:https://arxiv.org/html/2601.12626v1; PDF:https://arxiv.org/pdf/2601.12626; main results on pp.5–7, limitations pp.8–9, data/ablations Appendix A/B, Tables 1–2.
• ICLR acceptance and presentation:https://iclr.cc/virtual/2026/poster/10011690；OpenReview：https://openreview.net/forum?id=2zXRGiorSu。
• Official code:https://github.com/Raphoo/linear-mech-vlms。
• Kamath et al.：https://aclanthology.org/2023.emnlp-main.568/。
• Feng & Steinhardt：https://proceedings.iclr.cc/paper_files/paper/2024/hash/9d1b7fc578c0d2d6431fc26d736ecaf3-Abstract-Conference.html。
• Hernandez et al.：https://proceedings.iclr.cc/paper_files/paper/2024/hash/2b6d7285306fe896fca9b2e9a02ff6ec-Abstract-Conference.html。
• Zhang et al.：https://aclanthology.org/2025.naacl-long.115/。
• Chen et al. / AdaptVis：https://proceedings.mlr.press/v267/chen25cr.html。

Footnote: All numbers come from paper text, tables, or appendices actually read. Missing per-model video accuracy, exact p-values, seed variance, scale sweeps, and a complete same-protocol5×6 matrix remain missing, without pixel estimates or cross-paper fabrication.ml-report-index.md This entry has been updated. HTML passed warm-editorial-report the static validator(0 errors/0 warnings); the runtime lacks /usr/bin/chromium or /opt/pw-browsersand existing browser policy blocks local reports, preventing actual Playwright light/dark/mobile screenshots. To reduce risk, wide tables use horizontal-scroll containers and SVGs use responsive viewBox, with single-column mobile KPIs and narrower padding. These are static overflow checks, not actual browser acceptance testing.