Lumos: Can One Real Iteration Replace Trial and Error on 512 H100s?

0. Verified links and evidence conventions

These links were visited on 2026-09-04. Experimental numbers come only from PDFs read page by page; web pages verify versions, venues, and resource availability. No values were guessed from curve pixels: Figure 7 prints values directly on stacked bars; unlabeled bars and curves elsewhere support only qualitative judgments.

Evidence labels：Reported by the paper means the authors' text or table/figure labels;recalculated in this report means only sums, differences, and percentages;not reported by the paper is never filled in with speculation. HTML, Markdown, and plain-text versions were generated from one Markdown source and contain the same content.

1. Candidate pool, deduplication, and actual random selection

First consult ml-report-index.md: QServe (quantized serving), PARQ (quantization training), and the nearby generation-system topic LongLive were covered in depth within the past 14 days. They and near-duplicate low-bit topics were excluded. Candidates came from cross-searches of MLSys 2025 proceedings, OSDI 2025, arXiv, OpenReview, Hugging Face Daily Papers, Papers with Code, and Semantic Scholar. The latter four support discovery or verification, rather than replacing papers with secondary summaries.

Locally running date +%u returned 5confirmed Friday; running shuf -i 1-24 -n 1 returned 16selected Lumos as the core paper.Redraws: 0. The main text, appendix-level discussion, conference final version, and nine numerically labeled scale-out decompositions support an in-depth review. Published about 17 months before the report date, it remains within the 24-month limit. Main-conference systems papers were prioritized, and the random draw did not select for popularity.

2. Motivation: predicting when work can actually run is harder than counting FLOPs

Large-model training configurations form a Cartesian product of model architecture and system choices: layers, hidden size, FFN width, and micro-batches, multiplied by TP, PP, DP, pipeline schedules, collectives, and network topology. Simple analytical models add component times but cannot determine computation/communication overlap. Coarse trace simulators may miss causal edges between host threads, CUDA runtime calls, and streams.

The paper illustrates this with one GPT-3 175B iteration usingTP=8, PP=4, DP=8 . Figure 1 (PDF p.2; extracted web p.1, lines 60–65) gives dPRO/measured values for four components, beginning with exposed compute: 3235/4287 ms、overlapped 1691/885 ms、exposed communication 1417/2261 ms、other 81/178 ms. These sum to 6424 vs 7611 ms, meaning dPRO underestimates total duration by 1187 ms（15.60%）. More importantly, it underestimates genuinely exposed compute by 24.54%while overestimating overlap by 91.07%(recalculated here). An incorrect total can sometimes be patched with calibration; an incorrect decomposition directs engineers toward the wrong bottleneck.

This defines Lumos's problem: given one or a few real traces, construct a sufficiently faithful execution graph G=(V,E)that replays end-to-end makespan while retaining compute, communication, overlap, host/runtime behavior, and timing gaps. Then modify only the nodes, edges, and durations affected by a new configuration to obtain counterfactual estimates far more cheaply than deploying it.

The key assumptions and costs are:

1. Iteration stability: one or a few traces represent long-running training. The paper therefore calls profiling overhead negligible across training, but reports no overhead percentage (§5, PDF p.10, lines 760–767).
2. Transferable dependency patterns: local dependency templates from the original trace remain valid after changing layer count or PP (§3.4, PDF p.5, lines 360–383). This enables cheap transformations but may miss compiler choices, fusion, and congestion transitions.
3. Externally supplied durations: new GEMM/collective timings come from an internal model trained on fleet traces. The kernel predictor is explicitly outside the paper's scope (§5, PDF p.10, lines 783–797). Thus, the public method addresses when work can be scheduled, without fully disclosing how long new work takes.

3. Argument: from profiles to counterfactual execution graphs

Lumos has a tightly connected design: Kineto observes; dependency recovery restores causality; graph transformations propose counterfactuals; and discrete-event simulation executes them.

Layer 1: minimally intrusive collection. Kineto records PyTorch operators, CUDA runtime events, and GPU kernels, including start times, durations, stream/thread IDs, and correlation IDs. The paper says integration needs about 10 lines of hooks; the system contains approximately 5200 LoCof Python, and converting an original trace into a predicted trace takes seconds to minutes (§4, PDF p.6, lines 437–445). This is easier to deploy than older tools requiring framework/model modifications, but inserting hooks into source code is still integration work.

Layer 2: restoring four dependency classes. There are only CPU-task and GPU-task nodes, but edges cover all four combinations (§3.3, PDF pp.4–5, lines 252–318):

• CPU→CPU: sequential edges within a thread and cross-thread blocking edges inferred from significant execution gaps;
• CPU→GPU: correlation IDs connect cudaLaunchKernel/cudaMemsetAsync to the corresponding kernel;
• GPU→CPU：cudaDeviceSync/cudaStreamSync connects completed kernels back to blocked host tasks;
• GPU→GPU: sequential edges within a stream; across streams, pair cudaEventRecord with cudaStreamWaitEvent。

The substantive advance over dPRO is explicit preservation of cross-stream happens-before relations. LLM training frequently uses independent streams to overlap NCCL and GEMM. Missing a wait edge makes the simulator assume simultaneous execution, inflating overlap as in Figure 1.

Layer 3: graph transformation. For DP changes, local computation stays fixed and only collective durations change. For PP changes, layers are reassigned to stages under a 1F1B schedule, communication nodes inserted, and adjacent dependencies rebuilt. Changing depth duplicates layer subgraphs; changing hidden/FFN widths updates tensor shapes and critical GEMM/communication durations (§3.4 and §4.3.2, PDF pp.5,9, lines 360–383 and 717–726). The paperdoes not support changing TP, arguing that TP is usually fixed within a node in practice and leaving it for future work (PDF p.5, lines 370–373).

4. Formalization, pseudocode, and workflow

Let each task v∈V have processor p(v)(a CPU thread or CUDA stream), duration d(v), fixed predecessors Pred_f(v) , and runtime predecessors Pred_r(v,t). Its simulated start time can be written as:

s(v) = max( A[p(v)],  max_{u∈Pred_f(v)∪Pred_r(v,t)} (s(u)+d(u)) )
f(v) = s(v)+d(v)
A[p(v)] ← f(v)
T_iter = max_v f(v) − min_v s(v)

A[p] is the processor's next available time. Fixed edges count toward indegree at initialization; runtime edges are resolved only when a task is selected. For example, cudaStreamSync must wait for the last kernel in that stream at that moment, which need not be known before simulation. Algorithm 1 (PDF p.6, lines 384–417) can be rewritten as:

R ← all tasks with fixed indegree 0; A[p] ← 0
while R is nonempty:
    v ← pick(R)
    Discover v's runtime dependencies and attach v to its predecessors' dependents
    if v currently has indegree 0:
        s(v) ← max(A[p(v)], known earliest start time)
        A[p(v)] ← s(v)+d(v)
        for c in dependents(v):
            indegree(c) -= 1
            earliest(c) ← max(earliest(c), A[p(v)])
            if indegree(c)=0: R.add(c)
Output a new trace with every task's start/end times

This is more than measuring once and multiplying by a scaling factor:

                                       ┌─ CPU thread order / blocking ─┐
Kineto trace ─parse─┼─ Launch / synchronization edges ┼─→ G=(V,E)
                   └─ CUDA stream / event edges ───┘       │
                                                           ├─ Original replay → accuracy audit
New DP/PP/depth/width → Copy/split graphs, add edges, update durations
                                                           └─ Event simulation → predicted trace
                                                                            ├─ Makespan
                                                                            ├─ Exposed compute/communication
                                                                            └─ Overlap/activity timeline

5. Key mechanisms and engineering details

6. Data, platform, benchmarks, and metrics

This is a systems-trace study, not a downstream ML dataset evaluation. Models use the open-source NVIDIA/MLPerf Training GPT-3 implementation. Table 1 (PDF p.7, lines 467–471) specifies four architectures: 15B=48 layers / 6144 hidden / 12288 FFN / 48 heads，44B=48/12288/24576/48，117B=96/12288/24576/96，175B=96/12288/49152/96, all with head dimension 128.

The platform supports up to 512 H100 GPUs, 32 servers, 8×400 Gbps per host, and a RoCE data-center network. Software versions are CUDA 12.4, PyTorch 2.5, Transformer Engine 0.12.0, and PyTorch Lightning 1.9.4 (§4.1, PDF p.6, lines 456–460). This is substantial scale, but one GPU/network generation leaves extrapolation to A100, Blackwell, InfiniBand, or congested shared clusters untested.

The main metric is absolute relative iteration-time error:Err=|T_pred−T_real|/T_real×100%. Time is decomposed into exposed compute, overlapped time, exposed communication, and other. The paper also uses the term SM utilization, but defines it as the fraction of time within each 1 ms window during which at least one CUDA stream executes a task (§4.2.3, PDF p.8, lines 615–622). This is not CUPTI active warps, SM occupancy, tensor-core utilization, or achieved FLOPS. A more accurate term is stream-active duty cycle; this report does not interpret it as hardware SM saturation.

7. Complete main-results matrices

7.1 Replay: Lumos versus dPRO

To avoid misinterpretation: mean error falls to about 0.236 timesdPRO's error. Training is not 4.24× faster: Lumos is a predictor, and the paper does not accelerate the training job itself.

7.2 Nine counterfactual scale-out configurations × four time components

The table transcribes every labeled value in Figure 7 (PDF p.8, web lines 629–675), in ms;P/A denotes Lumos predicted/actual values. Total errors are recalculated from printed integers and may differ slightly from averages computed using unrounded data.

Per-group results are more informative than the mean 3.3% headline. Errors for the three DP-only points are 0.62%–2.98%; for the three PP points, 2.88%–6.83%; among joint expansions, 2×8×8 reaches 7.76%. The authors separately report a mean error for simultaneous DP+PP scaling of 4.2%(§4.3.1, PDF p.9, lines 697–700), versus 4.01%when recalculated from the three printed bars. The difference of 0.19 pp likely reflects rounding and does not establish a contradiction.

Component errors can be large even when the total is close:2×4×8 has predicted other time of 55 versus measured 39, a relative component overestimate of 41.03%. Because it is a small component, total error is still only 2.74%. Lumos's strongest evidence concerns makespan prediction. Claiming equal accuracy for every bottleneck requires a component-wise error matrix, which the paper does not summarize.

7.3 Model-architecture changes

Table 2 (PDF p.9, lines 703–709) derives four variants from GPT-3 15B: V1 20B (64 layers), V2 30B (96 layers), V3 28B (hidden 9216 / FFN 18432), and V4 44B (hidden 12288 / FFN 24576), all using TP=2,PP=2,DP=4. Figure 8 shows similar actual and predicted decompositions, butthe bars have no exact labels, and the text reports neither mean nor maximum error. This report therefore supplies no missing numbers and cannot establish whether changing depth or width is more reliable.

8. Layered ablations, hyperparameter sensitivity, and transitions

8.1 Layered ablations

The paperdoes not report a standard component-ablation table: it does not separately remove CPU→GPU correlations, GPU→CPU synchronization, inter-stream events, or runtime dependencies. Figure 6 presents only one Actual/Lumos/dPRO GPT-3 15B, TP2-PP2-DP4 activity timeline without a numerical distance metric. Thus, the complete system outperforms dPRO, but the 10.7 pp decrease cannot be quantitatively assigned to one edge class.

Figure 1 supports a mechanism-consistency check: dPRO overestimates overlap by 806 ms, and Lumos explicitly restores inter-stream happens-before relations. This supports the causal narrative but is not a controlled ablation. The most valuable missing experiment would remove each dependency class within Lumos and report makespan, component L1 error, and critical-path edge recall across nine configurations.

8.2 Hyperparameter sensitivity and transitions

The paperNot reported: sensitivity to the gap threshold, number of profiled iterations, the 1 ms utilization bin, ready-task pick selection rules, or kernel-predictor errors. Two potential transitions are particularly important:

• If communication-predictor error changes collective completion order, the critical path may switch discretely; end-to-end error need not grow linearly with single-kernel error.
• If changing PP stages triggers different kernels, fusion, or memory strategies, copying an existing layer subgraph may leave its transferable regime.

The paper tests DP/PP changes from a 16-GPU baseline to 32/64/128 GPUs (§4.3.1, PDF p.9, lines 678–700), without reverse scale-down, different GPU models, or congested workloads. It supports a low-error local neighborhood around the tested H100/RoCE implementation, rather than a smooth global configuration space.

8.3 Statistical significance

The paperNot reported: repetition counts, standard deviations, confidence intervals, significance tests, or variance over time. Neither cluster-contention distributions nor profiler run-to-run variation are given. Thus, 3.3% and 4.2% are descriptive statistics on selected configurations, not probabilistic guarantees before deployment. If iteration times fluctuate by 3%–5%, single-point prediction error may be comparable to measurement noise; the paper cannot separate modeling error from system noise.

9. Contributions, reusable designs, and transfer potential

Contribution 1: representing host, runtime, and streams in one causal execution graph. This abstraction is reusable: an optimizer can modify DAG nodes, edges, or durations without understanding all of PyTorch. It also explains why simply adding operator and NCCL times fails.

Contribution 2: predictions remain traces. The output is not an isolated iteration-time number: it supports further analysis of exposed communication, overlap, and activity timelines. This enables a diagnostic loop: identify the critical path, propose kernel/parallelism changes, replay them on the graph, and implement only the most promising candidates.

Contribution 3: replacing redeployment with local graph modification for configuration exploration. DP modifies communication; PP reorganizes layers/stages; width/depth changes replace affected key kernels. This is faster than building analytical models from scratch and cheaper than dedicating hundreds of GPUs to each candidate.

Transfer to inference, MoE, heterogeneous accelerators, or elastic training requires three conditions: a profiler exposing both host and device events, formalizable dependency templates for new schedules, and a public calibrated source of kernel/network durations. Without all three, framework independence describes postprocessing code rather than end-to-end usability.

10. Related work: problem and technical lineage

10.1 Daydream: the origin of graph transformations

Problem addressed: Daydream constructs kernel dependency graphs from low-level CUPTI traces and uses domain knowledge plus transformation primitives to predict optimization effects before implementation, as its USENIX page explicitly states. Presentation materials report five optimization classes on five DNNs, with mean estimation error around 8%（presentation p.3）。

Remaining gap: framework/model instrumentation is required, and the focus is single-machine / earlier-DNN optimization rather than complex host/runtime and cross-stream relations in modern distributed LLMs.Lumos's response: retain trace→DAG→graph transform→simulate, replace collection with Kineto, and restore four edge classes.Transferred design: Daydream's principle of modifying the graph before modifying the system directly precedes Lumos's what-if interface.

10.2 dPRO: global distributed graphs and a comparable baseline

Problem addressed: dPRO merges fine-grained communication traces from multiple workers into a global data-flow graph and searches computation/communication/memory optimizations. Its paper reports prediction error in most settings of <5%and optimization gains up to 3.48×(arXiv abstract; MLSys 2022 PDF p.1).Remaining gap: it targets the TensorFlow/MXNet and AllReduce/parameter-server era. Lumos identifies missing concurrent-/inter-stream dependencies, producing 14% mean error on new LLM configurations.

Lumos's response: explicitly recover cross-stream causality using event pairs and runtime synchronization, reducing mean error to 3.3% in the same study.Transferred design: dPRO's global graph, topological simulation, and optimization hooks remain; Lumos primarily upgrades trace granularity and dependency semantics. Note that dPRO's original <5% and Lumos's remeasured 14% use different models, frameworks, and hardware; they do not make the earlier paper self-contradictory.

10.3 TraceSim: Lumos's direct prototype

Problem addressed: TraceSim, from the same author lineage, already uses Kineto to preserve operators, runtime calls, and kernels together. On GPT-3 with 256 H100s, it reports mean replay accuracy of 95.6%and scale-up accuracy up to 99.5%（PDF p.1 lines 13–19；p.5 lines 423–432）。Remaining gap: the five-page workshop paper is a proof of concept, with limited treatment of PP/architecture transformations, dynamic-dependency algorithms, and limitations.

Lumos's response: expand to 512 H100s, four GPT-3 sizes, an explicit Algorithm 1, nine DP/PP counterfactuals, and architecture variants.Transferred design: multilayer Kineto traces, four dependency classes, and predicted-trace output transfer almost unchanged. Lumos is therefore better understood as a mature, systematized, more extensively validated version than an entirely different method.

10.4 ASTRA-sim 2.0: a complementary source for communication and hardware exploration

Problem addressed: ASTRA-sim 2.0 supports arbitrary parallelism with a graph-based training loop and models multidimensional heterogeneous topologies, collectives, in-network computation, and disaggregated memory (arXiv v1, pp.1–2).Remaining gap: it requires workload and system abstractions rather than automatically recovering complete host/CUDA happens-before relations from one PyTorch run.

Lumos's response: Lumos extracts workload execution graphs from real traces. Conversely, it uses internal models for unseen collective durations and explicitly proposes future integration with ASTRA-sim (§4.3.1/§5, PDF pp.9–10).Transferred design: ideally Lumos supplies real causal structure and ASTRA-sim supplies communication durations under new topologies. They are complementary rather than substitutes.

10.5 Calculon: the analytical-search branch

Problem addressed: Calculon parameterizes models, hardware, and implementation strategies to explore joint LLM software/hardware spaces rapidly under constraints such as memory. Its open-source README gives approximately 1 secondper configuration; its execution optimizer for 5128 A100 GPUs takes about 1 minute。Remaining gap: analytical models rely on idealized decompositions and struggle to preserve dynamic host launches, stream waits, and real runtime gaps.

Lumos's response: anchor dynamic behavior in measured traces, sacrificing zero-shot prediction for entirely new hardware.Transferred design: Calculon suits coarse global screening, while Lumos suits precise local ranking near a deployed stack. Calculon can narrow the search before Lumos replays candidates.

10.6 Nonlinear technical evolution

Hardware / analytical models ─ Calculon ─────────┐
                                                 ├─ Coarse screening of new hardware / global designs
Network / collective simulation ─ ASTRA-sim 2.0 ─┤
                                                 └─ Duration supplier ─┐
                                                                       │
CUPTI kernel DAG ─ Daydream ─┐                                          │
                            ├─ Global distributed graph ─ dPRO         │
                            │                         │                │
                            └─ Graph transformations  │                │
                                                      ▼                ▼
                                    Kineto operator+runtime+kernel
                                                      │
                                    TraceSim: cross-stream dependency prototype
                                                      │  Scale / graph transforms / dynamic edges
                                                      ▼
                                         Lumos: local counterfactual replay
                                                      │
                            ┌─────────────────────────┴────────────────┐
                            ▼                                          ▼
                 Open kernel/network predictors              Probabilistic noise and OOM/energy

Drivers and turning points: Daydream first shifts profiling from explaining past execution to predicting unimplemented optimizations. dPRO then expands single-device graphs into global distributed graphs. Multistream computation/communication overlap in LLMs subsequently invalidates coarse edge semantics, prompting TraceSim/Lumos to model operator/runtime/kernel causality. The next step should be public duration models and uncertainty propagation; otherwise private latency estimates still constrain accurate local graphs.

10.7 Cross-paper comparison

11. Limitations acknowledged by the paper

1. Feasibility blind spots: counterfactual configurations are assumed to run successfully; OOM is not predicted. FLOPS utilization, memory, bandwidth, and energy are also outside scope (§5, Limitations, PDF p.10, lines 808–816). The fastest predicted configuration may not fit in memory.
2. New-kernel durations are outside scope: the current system relies on internal fleet-trace models. Communication could use ASTRA-sim/analytical models and computation could use microbenchmarks, but the paper does not integrate these or propagate their errors (§5, PDF p.10, lines 783–797).
3. TP cannot be changed: only DP/PP and model width/depth are supported, because the authors regard TP as generally fixed within a node (§3.4, PDF p.5, lines 370–373). This omits a key training-configuration search dimension.

12. Independent critique: evidence and implications

1. [Reproducibility] The paper describes roughly 5200 LoC but publishes neither code, traces, nor its internal kernel predictor. External researchers cannot reproduce 3.3% or distinguish graph-structure accuracy from private duration-model accuracy.
2. [Metric validity] SM utilization is defined as any-stream-active time within 1 ms (PDF p.8), so even low-occupancy kernels count as active. Figure 6 therefore establishes similar active/idle timelines, not efficient SM/Tensor Core utilization.
3. [External validity] All main experiments use H100s, RoCE, one GPT-3 implementation, and one software stack. The 3.3% error cannot be generalized to A100/Blackwell, InfiniBand, MoE, or different compilers.
4. [Ablation attribution] Without edge-class removal ablations, comparing full Lumos with dPRO cannot attribute the entire −10.7 pp mean-error change to inter-stream events. Profiler, parsing, and duration-predictor changes may also contribute.
5. [Statistical reliability] Without repeated runs, standard deviations, or intervals, 2×2×8 's 0.62% and 2×8×8 's 7.76% cannot be separated from natural cluster variation, leaving deployment risk unbounded.
6. [Search completeness] TP is fixed, PP examples use 1F1B/regular layer assignment, and layers are assumed uniform. Lumos is therefore a local configuration estimator, not a complete automatic 3D-parallelism planner.
7. [Feasibility / objective mismatch] OOM, energy, training costs, and throughput variation are unmodeled. Fastest predicted does not mean runnable, cheapest, or most energy-efficient, and cannot directly justify procurement decisions.
8. [Component credibility] Because 2×4×8 predicts the other component as 55 ms rather than 39 ms (+41.03%) while total error is only 2.74%, low makespan error can hide large relative errors in small components. Bottleneck diagnosis requires component-wise error reporting.

13. Open challenges: actionable head-to-head experiments

Theory

• Dependency-edge error propagation: compare full Lumos, event-edge removal, host-sync removal, and random removal of 1/5/10% of edges on the same trace. Evaluate critical-path precision/recall, makespan error, and component L1 error to bound the impact of missing each edge class.
• Uncertainty propagation: compare point-estimate Lumos with probabilistic Lumos assigning a duration distribution to each kernel. Across 30 real runs on different days, evaluate 50/90/95% prediction-interval coverage rather than only mean absolute error.

Engineering applications

• Public duration suppliers: hold the Lumos graph fixed and connect microbenchmark lookup, Calculon/AMPeD, ASTRA-sim, or learned kernel predictors. Compare them with the internal model on the same nine configurations, reporting accuracy, calibration cost, simulation time, and required real GPU-hours.
• Budget-matched configuration search: give Lumos, Calculon, and Megatron empirical grid search the same 24-hour wall-clock / H100-hour budget to find the fastest non-OOM configuration over TP×PP×DP×microbatch. Reevaluate the top five on real hardware, measuring best-found throughput and failure rate.
• Cross-platform transfer: collect source traces for the same model on H100-RoCE, A100-InfiniBand, and B200-NVLink, then predict the other two platforms. Compare with reprofiling on every platform to quantify trace transferability and cold-start cost.

Emerging directions

• Dynamic MoE routing: compare static single-iteration Lumos with a stochastic simulator retaining token-routing distributions on an MoE with controllable expert imbalance. Increase skew progressively and measure p50/p95 iteration times and straggler attribution.
• Compiler transitions: vary hidden size to trigger different Triton/cuBLAS kernels or fusion choices. Compare old-graph copying plus duration updates with recompilation plus reprofiling, identifying thresholds at which graph topology changes.
• Joint training–energy objectives: attach NVML power traces to candidates with equal throughput. Compare makespan-only optimization with Pareto search over time, peak memory, and kWh, testing whether 3.3% timing error preserves Pareto rankings.

14. Recent-work radar (not core evidence)

1. vTrain(2023/2024): simulates cost and compute-optimal LLM training; useful for comparing cold-start costs of beginning with abstract workloads versus Lumos's real traces.
2. Proteus(EuroSys 2024): expresses complex parallelism using strategy trees, complementing Lumos's fixed TP. Compare budget-matched searches over the same Megatron space.
3. ReaLLM(ASAP 2025): connects chip-level kernel libraries to trace-driven system inference, reporting 9.1% mean latency error on 4×H100. Its scope is inference rather than training.
4. PRISM(2025): represents system variation as distributions rather than fixed durations, directly addressing Lumos's lack of confidence intervals. Public implementation status remains subject to the authors' stated release plans.
5. Simulating LLM training workloads for heterogeneous compute(2025): introduces nonuniform workload partitioning, useful for testing when Lumos's uniform layer/stage assumptions fail in heterogeneous clusters.
6. TileSight(2026): models GPUs from tile-first principles, reconnecting low-level analysis with high-level systems. New-kernel durations remain its central complement to Lumos.
7. Charon(MLSys 2026): unifies training and inference through compiler-style layer-by-layer lowering for fine-grained simulation. The authors report overall error below 5.35% and large-training error below 3.74%, making it a recent direct head-to-head comparator for Lumos.
8. MLCommons Chakra(MLSys 2026 Industry Track): defines an open execution-trace exchange format for compute, memory, communication, control/data dependencies, and resource constraints. It may remove proprietary-format barriers between Lumos traces and other simulators.

15. Conclusion: what merits attention and what remains unsupported

Lumos's most reusable idea islocal counterfactual analysis grounded in real happens-before relations. It shows that modern LLM training prediction errors often arise from lost causal edges among hosts, runtimes, CUDA streams, and synchronization, rather than insufficiently precise FLOPs equations. Reducing mean replay error from 14.0% to 3.3% (−10.7 pp, −76.43%) is strong evidence. Yet the nine scale-out cases span 0.62%–7.76% error, rather than guaranteeing 3.3% universally.

The deployment assessment must remain measured. Around the authors' H100/RoCE/PyTorch/GPT-3 stack, Lumos is a promising configuration sandbox. Without a public implementation and kernel predictor, TP transformations, OOM/memory/energy models, and statistical intervals, it is not an independently reproducible full-stack capacity planner. The most useful next step is a verifiable integration of Lumos graphs, ASTRA-sim communication, public kernel predictors, and probabilistic error propagation, rather than simply adding a more complex DAG.

Verification notes: this report read the 13-page Lumos arXiv v1 / MLSys final paper and five-page TraceSim paper, and visited conference, arXiv, code, and project entries for related work. The original candidate table is saved as 2026-09-04_efficiency_candidates.tsv; the random draw returned 16, with no redraw. Missing ablations, standard deviations, significance tests, exact Figure 8 values, and profiling overhead remain explicitly unreported.ml-report-index.md was updated with this entry. The HTML passed the Warm Editorial structural validator (0 warnings) and static DOM/responsive checks. Playwright is installed, but the environment lacks a browser executable at /usr/bin/chromium or /opt/pw-browsers ; as instructed, playwright installwas not run. Browser screenshot checks in light, dark, and mobile viewscould not be performed in this runand are not claimed complete.