DEFAME: Turning Multimodal Fact-Checking into a Dynamic Evidence-Gathering Loop

0. Verified links and evidence boundaries

These URLs were visited on 2026-09-08. Numbers come only from the ICML/PMLR paper and arXiv v4 HTML/appendices read during this run; indexes verify venues, versions, and assets. The core arXiv paper 2412.10510v4，2025‑07‑24; PMLR lists ICML 2025, pp.5383–5417. OpenReview triggers browser verification, confirming an entry but not readable evidence. Exact-title searches in Papers with Code and Semantic Scholar did not return stable core entries; this means independent records could not be verified here, not that they are absent from those indexes.

Evidence references: arXiv v4 lines 192–203 refer to the line-verifiable HTML above; Tables 3/4/5/8 use the paper's numbering. Percentage-point and relative changes are explicitly labeled recalculations from original tables; no numbers are guessed from figures.

1. Candidate pool, deduplication, and actual random selection

First read ml-report-index.mdto exclude Visual Graph Arena, Q-VDiT, DocVXQA, AVUT, LongLive, DreamGen, and adjacent topics covered in the preceding 14 days. Candidates came from ICML 2025 PMLR, ICLR 2026 OpenReview, EMNLP/ACL Anthology, and recent arXiv, with Hugging Face Daily Papers, Papers with Code, and Semantic Scholar used for discovery and cross-searches. The full list is saved as candidates_2026-09-08.tsv. Running shuf -i 1-23 -n 1returned 4, selecting DEFAME. Official text, appendices, code, and data are accessible, so no redraw was needed。

1. Unifying 2D and 3D Vision-Language Understanding（ICML 2025）
2. Do Vision-Language Models Really Understand Visual Language?（ICML 2025）
3. ELITE: Enhanced Language-Image Toxicity Evaluation for Safety（ICML 2025）
4. DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts（ICML 2025）
5. Improving Zero-Shot Adversarial Robustness in Vision-Language Models（ICML 2025）
6. Multimodal Time-Aligned Shared Token Recommender（ICML 2025）
7. CoreMatching: Co-adaptive Token and Neuron Pruning for VLMs（ICML 2025）
8. Agent Reviewers: Domain-specific Multimodal Agents with Shared Memory（ICML 2025）
9. Empowering World Models with Reflection for Embodied Understanding（ICML 2025）
10. video-SALMONN-o1: Reasoning-enhanced Audio-visual LLM（ICML 2025）
11. The Hidden Life of Tokens: Reducing Hallucination via Visual Information Steering（ICML 2025）
12. Compositional Condition Question Answering in Tabular Understanding（ICML 2025）
13. Orthus: Autoregressive Interleaved Image-Text Generation（ICML 2025）
14. Be Confident: Uncovering Overfitting in MLLM Multi-Task Tuning（ICML 2025）
15. Training Vision-Language Process Reward Models for Test-Time Scaling（ICLR 2026）
16. Multi-modal Data Mixtures for Vision-Language Model Training（ICLR 2026）
17. RobustVLA: Robustness of Vision-Language-Action Models（ICLR 2026）
18. VSTAT: Benchmarking Visual State Tracking（arXiv 2026）
19. HAVEN: Hierarchically Aligned Multimodal Benchmark（arXiv 2026）
20. MVU-Eval: Towards Multi-Video Understanding Evaluation（arXiv 2025）
21. Video-MMLU（arXiv 2025）
22. MT-Video-Bench（arXiv 2025）
23. MMIU: Multimodal Multi-image Understanding（ICLR 2025）

2. Motivation: why looking at an image and directly judging truth is insufficient

Real-world fact-checking is not static classification. Claims may combine text, photographs, screenshots, or composite images, while evidence is scattered across web text and images. About 80% of professionally fact-checked claims are multimodal(introduction, arXiv v4 lines 75–79). Existing automated systems are largely text-based; many visual approaches reduce images to captions or rely on parametric memory, failing to handle both multimodal claims and external multimodal evidence (lines 79–82). This creates three problems: events after the knowledge cutoff require new evidence; explanations without sources cannot be audited; and one-shot retrieval cannot pursue follow-up questions.

Old benchmarks can create false reassurance. AVeriTeC ends in 2021-12, VERITE spans 2001-01 to 2023-01, and MOCHEG ends in 2022-05, whereas the tested GPT-4o has a 2023-10 knowledge cutoff. Public fact-check pages may have entered pretraining (Appendix A, lines 418–433). ClaimReview2024+ therefore contains300 English claims from articles dated 2023-11-01 through 2025-01-18, comprising 160 text-only and140 text–image claims. Labels are Refuted 129, Supported 89, Misleading 61, and NEI 21 (§4, lines 165–174). New articles can revisit older events, as the authors acknowledge, so this is not a guarantee of unseen events, but is closer to temporal out-of-distribution evaluation than pre-cutoff static sets.

Table 3 provides direct evidence: GPT-4o scores on older AVeriTeC/MOCHEG/VERITE-T/F are 62.1/53.7/78.7, but ClaimReview2024+ falls to 35.2and CoT falls further to 31.4, versus DEFAME's 70.5/59.2/83.9/69.7(arXiv v4 lines 192–203). Reasoning capability and access to current evidence are separate variables; for new events, evidence gathering matters more than generating additional reasoning steps.

3. Argument: from a static classifier to an evidence process with a stopping rule

The paper does not introduce a new visual encoder; it integrates fact-checking subtasks into a dynamic loop. One MLLM, four external tool types, and a growing fact-check report form the system. Every call sees the current report and stage instructions (§3, lines 127–132). The report serves as working memory, an evidence ledger, and material for the final explanation.

The six stages are:

1. Plan： Generate actions for missing information, recording completed actions to avoid repetition. Tools are Web Search, Image Search, Reverse Image Search (RIS), and Geolocation.
2. Execute： Web/Image Search each retrieve the top 3 pages; RIS retrieves up to 3 pages containing the input image; GeoCLIP proposes countries. Web results are restricted to dates preceding the claim and major fact-checking sites are excluded to reduce answer leakage (lines 133–153).
3. Summarize： Extract relevant text, images, and URLs; results that contribute nothing return NONE, compressing long pages into the report (lines 154–156).
4. Develop： Reason step by step over the claim and summarized evidence, explicitly identifying gaps (lines 157–159).
5. Judge： Output a dataset label. NEI returns to Plan; stop at a determinate label or after at most 3 iterations (lines 160–161).
6. Justify： Condense the full report into a short explanation with hyperlinks, rather than presenting the entire investigation trace to users (lines 162–164).

Four assumptions matter: top-ranked search results provide sufficient recall; the MLLM identifies relevant evidence and resists webpage prompt injection; errors across planning, summarization, reasoning, and judging do not reinforce themselves; and NEI reliably signals whether retrieval should continue. The dynamic loop improves coverage and auditability but extends reproducibility dependencies from a fixed model to indexes, APIs, web states, scrapers, and date filters.

4. Formalization and executable pseudocode

Let the text space be T, image space I, multimodal sequence space M=(T∪I)*, and label space Y. DEFAME is:

F: M → M × Y,       F(c) = (R_out, y_out)                         (1)

where input claim c may interleave text and images, andR_out is the full report. An iteration composes five stages (Appendix F, lines 520–553):

(R^(i+1), y^(i+1)) := F_iter(R^i)
F_iter := S5 ∘ S4 ∘ S3 ∘ S2 ∘ S1                               (2)

A_i = S1(R^i)                             # Plan actions
E_i = { τ(a) | a ∈ A_i }                  # Execute tools and collect evidence
R1^i = σ(E_i, R^i)                        # Append contextual summaries
R2^i = S4(R1^i)                           # Develop evidence-based reasoning
(R3^i, y^i) = S5(R2^i)                    # Intermediate verdict

The stopping time is:

i* = min { i ≤ N | y^i ≠ NEI or i = N },    N = 3
y_out = y^(i*)
R_out = S6(R^(i*))                                                (3)

This reveals a subtle issue: the stopping criterion is y≠NEI , but confidence in Supported/Refuted/Misleadingis not explicitly modeled. A premature Supported decision based on a superficially matching image can therefore stop the loop. Failure analysis indeed finds such cases (§5.7, lines 282–285; Appendix O, lines 706–710).

procedure DEFAME(claim c, max_rounds = 3):
    R ← init_report(c)
    for round in 1..max_rounds:
        A ← MLLM.plan(c, R, unused_tools_and_queries)
        E ← parallel_execute(A)           # web/image/RIS/geolocation
        E ← filter_by_date_and_domain(E)
        S ← MLLM.summarize_relevant(E, R) # irrelevant → NONE
        R ← append(R, A, S, URLs, images)
        R ← MLLM.develop(c, R)
        y ← MLLM.judge(c, R, label_rules)
        if y != NEI: break
    j ← MLLM.justify(c, R, y, require_hyperlinks=True)
    return (R + j, y)

5. Key mechanisms and engineering details

The public ICML branch provides Docker/manual installation,config/icml_experiments, batch scripts, and four paid/free API types. CPU-only execution requires no local LLM and a disabled geolocator. Reproduction also needs OpenAI, Serper, Google Vision, and Firecrawl/Hugging Face credentials, some paid (ICML-branch README, accessed lines 274–334). These define reproducibility more accurately than code availability alone.

6. Datasets, labels, and metrics

Accuracy enables uniform comparison, but CR+'s 129/89/61/21 imbalance means 69.7% cannot substitute for macro-F1, per-class recall, or calibration. Confusion matrices are provided without all per-class numbers tabulated in the text; this report does not estimate them from images.

7. Complete main-results matrices

Table 3 contains four datasets and six comparable metrics, not five benchmarks × six methods. The sparse original matrix is preserved; — indicates evaluation omitted because code was unavailable or settings incompatible.* denotes values derived in the original paper from existing results; bold marks column-wise best.

Source: core Table 3, arXiv v4 lines 192–204. The final three rows are means ± standard deviations over three runs. Recalculation against the strongest non-DEFAME competitor in each column:

The claim of surpassing all methods on all three old benchmarks needs qualification. DEFAME leads AVeriTeC, MOCHEG, VERITE T/MC, and T/F, but not VERITE T/OOC. It is better described asstrongest overall across tasks, with pronounced temporal generalization benefitsrather than first in every column.

8. Backbone replacement: modular does not mean backbone-independent

Source: Table 2, lines 185–191. Replacing GPT-4o with Llama 4 reduces CR+ by 20.9 pp（−30.0%）and VERITE by 11.6 pp（−13.8%）; LLaVA-1V reduces CR+ by 37.1 pp（−53.2%）. LLaVA often fails to format actions correctly, reducing actual tool calls (lines 205–208; Appendix C, lines 473–474). Plug-and-play is an interface property, not performance equivalence; structured-instruction compliance is itself an experimental variable.

9. Layered ablations: evidence coverage matters more than extra reasoning

Source: Table 4, lines 228–262; Δ recalculated here. Task dependence is clear: MOCHEG/CR+ are most sensitive to Web Search, while visual-mismatch VERITE is most sensitive to RIS. Multiple rounds improve MOCHEG over a single turn by 11.5 pp（+24.1%）but VERITE by only 1.1 pp（+1.33%）. Follow-up value depends on missing external facts, not a general superiority of looping.

Develop contributes only 0.1 ppon VERITE, far below RIS's 10.2 pp. This challenges attributing most gains to longer reasoning. A better attribution is:Evidence identifiability provided by the right tools > intermediate natural-language reasoning. Ablation tables lack standard deviations and may use one run, however, so differences of 0.1–1.0 pp should not be treated as robust.

10. Hyperparameter sensitivity, cost, and transitions

The paper fixes temperature at 0.01、top‑p 0.9, images per page at most 32 , rounds at most 3 , and search results per action at most 3 (five for AVeriTeC). No grids cover temperature, top-p, iterations, retrieval k, the 32-image cap, context truncation, or confidence thresholds. Robustness and statistical hyperparameter transitions therefore remain unknown.

Only two structural transitions are observable. First, modality: RIS contributes 10.2 ppon VERITE but only 1.0/5.7 pp on MOCHEG/CR+, whereas Web Search contributes 17.2 ppon MOCHEG. Second, backbone: LLaVA-1V action-format failures reduce search counts to 0.9/claim. Faster execution reflects missed tool calls, not more efficient reasoning.

Source: Appendix Table 5, lines 455–475. VERITE timing is approximate and highly variable due to shared hardware. Full DEFAME is about 42.3×slower than direct GPT-4o (254s/6s), with about 21.8×as many input tokens and at least roughly an order of magnitude greater LLM API cost, in exchange for VERITE T/F +5.2 pp. Removing Planning increases queries from 2.2 to 5.2 (+136.4%), input tokens from 48K to 70K (+45.8%), and still lowers accuracy by 0.5–1.7 pp. Planning's main value is cost control rather than a large standalone accuracy gain.

11. Statistical significance and explanation quality

Main results report three-run means ± SD for DEFAME, GPT-4o, and GPT-4o CoT, but no paired per-example tests, confidence intervals, or multiple-comparison corrections. CR+'s +34.5 pp is large relative to SDs, while significance of VERITE T/F's +3.9 pp over CoT on paired examples is unreported and cannot be inferred from SDs alone.

Human evaluation covers 36 claims and 185 ratings: 154 for DEFAME, 31 for CoT, with at least five per claim. All participants have higher education. A Mann–Whitney U test for completeness yieldsp≈9.1×10^-9, favoring DEFAME; consistency does not differ significantly (§5.6, lines 268–281; Appendix L, lines 683–688). The 154:31 imbalance, missing effect sizes, rater counts/agreement, and lack of separation between link presence and actual support mean this establishes preferred completeness, not faithful citations.

Automatic explanation metrics are mixed. Table 8 gives DEFAME ROUGE-L/BLEU/BERTScore of 18.72/3.20/85.89and, with one style example, 25.37/7.31/87.42, relative gains of +35.5%/+128.4%/+1.78%. Yet one-shot BLEU remains below an untuned baseline's 7.38 and BERTScore below MetaSumPerceiver's 88.10. The authors attribute large shifts mainly to n-gram/wording sensitivity rather than sudden explanatory improvement (Appendix M, lines 691–701).

12. Contributions, reusable designs, and transfer potential

Contribution 1: retain multimodal evidence throughout the investigation loop. Where RAGAR converts images to descriptions, DEFAME preserves interleaved text/images in claims, retrieved pages, reports, and subsequent reasoning, enabling direct comparison of RIS/geolocation results with the original image.

Contribution 2: adapt depth through NEI. Static pipelines follow the same path regardless of difficulty. DEFAME continues only when evidence is insufficient; planning reduces queries from a static 5.2 to 2.2 while improving accuracy.

Contribution 3: isolate retrieval value with temporal evaluation. CR+ is imperfect, but exposes direct GPT-4o's 35.2 accuracy despite stronger old-benchmark scores, making external evidence's role clearer than another old-benchmark SOTA claim.

Reusable designs: An append-only report state; action deduplication; explicit rejection of irrelevant results; retrieval bounded by claim timestamps; tools saved alongside clickable sources; and separation of lengthy investigation traces from concise explanations. These transfer to medical evidence review, incident investigation, product compliance, and scientific-literature auditing.

Transfer boundaries: Medical/legal applications need source hierarchies and conflict resolution rather than trusting search rankings. Video verification needs transcripts, keyframes, and provenance timelines. Multilingual systems need cross-language retrieval/translation tests, and real-time systems need budget-aware stopping and human escalation for high-risk cases.

13. Related work: problem and technical lineage

RAGAR（FEVER 2024）

Problem addressed: CoRAG/ToRAG uses MLLMs to ask sequential subquestions about political text–image claims; RAGAR reports weighted F1 of 0.85, exceeding baseline reasoning by 0.14(RAGAR arXiv abstract, accessed).Remaining gap: Images are verbalized first, retrieval is text-only, and the focus is politics.DEFAME's response: Retain original images, add image search/RIS/geolocation, and evaluate four substantially different benchmarks.Transferred design: Recursive follow-up determined by prior evidence becomes the NEI→Plan loop.

MOCHEG（SIGIR 2023）

Problem addressed: It first integrates retrieval, three-class fact-checking, and explanation generation in an end-to-end multimodal benchmark. The final version contains 15,601 claims, 33,880 textual evidence passages, and 12,112 evidence images (MOCHEG arXiv v3 abstract).Remaining gap: Its baseline is a trained, segmented pipeline with relatively fixed sources; explanation metrics are sensitive to surface overlap.DEFAME's response: Replace trained submodules with a zero-shot MLLM and dynamic tools, producing linked reports.Transferred design: DEFAME expands MOCHEG's retrieval–verification–explanation structure into six stages with iteration.

MetaSumPerceiver（ACL 2024 Main）

Problem addressed: Summarize multidocument text/image evidence specifically for each claim, reducing manual reading.Remaining gap: It relies on supplied evidence sets rather than open-world investigation, with MOCHEG verification accuracy of 48.6(DEFAME Table 3).DEFAME's response: Summarize after every retrieval and feed summaries into the next planning round.Transferred design: Evidence compression before judgment is retained, but summarization becomes a recurrent state-update operator σ(E,R)。

AVeriTeC（NeurIPS 2023 D&B）

Problem addressed: Provides 4,568 real claims from 50 organizations, with source-linked QA pairs describing evidence chains (official proceedings abstract).Remaining gap: Text-only, static, and earlier than modern MLLM cutoffs; development-set KB noise differs from the open Web.DEFAME's response: Validate text capability on its KB while adding CR+ for temporally shifted multimodal claims.Transferred design: AVeriTeC's four-label taxonomy and view of evidence as answerable gaps influence Judge rules and InFact adaptation.

SNIFFER（2024）

Problem addressed: Two-stage instruction tuning improves entity alignment and explanation of OOC differences; model weights are public.Remaining gap: Primarily parametric recognition/explanation, without open-domain multihop evidence chains.DEFAME's response: Retrieve provenance, location, and time through tools rather than training a specialized visual model.Transferred design: SNIFFER emphasizes that superficial text–image agreement does not establish shared context. DEFAME assigns this need to RIS and evidence reports.

Nonlinear technical evolution

                                                  ┌─ AVeriTeC: real claims + QA evidence ─┐
FEVER-style text checking ┤                                     ├─ InFact/DEFAME labels and evidence formats
                         └─ Planning-based LLM fact-checking ───┘

Text–image consistency ─ SNIFFER: OOC tuning ────────────────────┐
                                                               ├─ DEFAME: original images + RIS + GeoCLIP
MOCHEG: retrieve→verify→explain ─ MetaSum: multidocument summaries ┤       │
                                                               │       ├─ CR+: temporal evaluation
RAG / Agents ─ RAGAR: recursive CoRAG/ToRAG questions ──────────┘       │
                                                                       └─ 2026 dynamic benchmarks / multi-agent systems

Drivers and turning points: The first shift is from text–image similarity to externally verifiable evidence; the second, from fixed evidence bundles to the open Web; the third, from static benchmarks vulnerable to pretraining contamination to dynamically updated evaluation. DEFAME sits between the second and third: retrieval is dynamic, but CR+ remains a one-time snapshot.

14. Limitations acknowledged by the paper

1. External-evidence credibility: Search results may leak answers, be biased, or be unreliable. The system relies on diversity and search-engine ranking, without external source ratings (§6, arXiv v4 lines 286–290).
2. System stability: Access restrictions, long documents, and scraping failures cause variation; open MLLMs are sensitive to prompt formatting, and small changes may break action outputs (§6, line 291).
3. Hallucinations and human-evaluation scale: MLLM hallucinations need further study. Human evaluation covers only 36 claims and cannot span all failure modes (§6, line 292; Appendix L, line 688).

15. Independent critique: evidence and implications

1. [Attribution validity] Full DEFAME simultaneously changes retrieval, visual input, iteration depth, report memory, and prompt length. Single-component removal in Table 4, without compute-matched combinations, cannot attribute CR+'s +34.5 pp solely to multimodal evidence. Web-only, RIS-only, token-matched single-turn, and independent-judge factorial controls are needed.
2. [Universal SOTA wording] Table 3 gives VERITE T/OOC of 78.4, below AITR's 82.7. Surpassing all previous methods can therefore describe overall cross-benchmark capability, not winning every column.
3. [Stopping rule] Stopping requires only y≠NEI, and failure analysis finds premature Supported decisions after matching similar images. Label identity substitutes for confidence; stopping should instead consider evidence coverage, source independence, and calibrated probability.
4. [Self-confirmation bias] One MLLM summarizes, reasons, and judges, while later stages see its compressed report. Early omissions or misreadings become accepted facts. Independent evidence-entailment models or cross-model audits should interrupt this feedback.
5. [Search-ecosystem bias] The system excludes domains including Snopes and YouTube, yet an Appendix O shark-image case is verifiable only there. Anti-leakage rules systematically create missing evidence. Leakage detection should consider evaluation time and page content, rather than excluding entire domains.
6. [Labels and data quality] Of 20 inspected VERITE errors, the authors find 3 suspected mislabels (15% of that small sample); MOCHEG ground truth can also change over time. Point accuracy mixes model errors with benchmark aging. Dynamic verification should provide both claim-time and evaluation-time verdicts.
7. [Statistical completeness] Three main-result runs, no ablation SDs, and no paired tests cannot establish stable component rankings from 0.1–1.0 pp Develop/Planning/Geo differences. Freeze retrieval snapshots and apply multiseed bootstrap or McNemar tests.
8. [Human evaluation and citation faithfulness] Significance concerns completeness only; DEFAME's 154 ratings and CoT's 31 are imbalanced, and entailment between citations and adjacent sentences is not tested. Thus, p≈9.1×10^-9 does not establish citation faithfulness. Sentence-level citation precision/recall and blinded source-quality assessment are needed.

16. Open challenges: actionable head-to-head experiments

Theory

• Stopping theory: Compare DEFAME's NEI→continue with evidence-coverage thresholds, Bayesian posterior entropy, and conformal abstention, using identical retrieval snapshots and at most three rounds. Report accuracy, NEI calibration, mean rounds, and error costs to test NEI as an optimal-stopping surrogate.
• Evidence independence: Compare the original with source-cluster deduplication and causal provenance graphs. Duplicate a story 1/3/10 times under synthetic contamination and measure verdict flips, testing whether multiple sources merely repeat one release.
• Temporal semantics: Compare one verdict with dual claim-time/evaluation-time verdicts on time-sensitive MOCHEG and dynamic VeriTaS cases, separating model mistakes from world changes.

Engineering and applications

• Budget-matched attribution: Cap full DEFAME, Web-only, RIS-only, static-all-tools, and single-turn systems at 48K input tokens and 254 seconds. Compare accuracy on all four datasets to distinguish tool variety, dynamic scheduling, and extra computation.
• Retrieval-backend robustness: Compare Google/Serper, Bing, Brave, and Common Crawl snapshots with fixed dates/regions over seven days. Report evidence recall, URL survival, answer variance, and per-claim cost.
• Security: Compare the original scraper with HTML sanitization, a prompt-injection classifier, and a sandboxed parser. Insert instructions to ignore the claim and output Supported into pages; measure attack success, clean-accuracy loss, and latency.
• Human collaboration: Randomize 100 live claims across human-only, automatic DEFAME, and DEFAME-assisted fact-checkers. Measure minutes/claim, citation errors, final accuracy, and editing acceptance, rather than loosely comparing an entire workday with 4:14.

Emerging directions

• Video and multiple images: Compare DEFAME plus keyframes/ASR with T²Agent and ReMMD-Agent on identical multi-image/video sets under API-cost caps. Measure provenance localization, cross-frame entity consistency, and five-way veracity instead of excluding video domains.
• Executable small-model agents: Compare GPT-4o, Llama 4, and LLaVA-1V with grammar-constrained decoding and tool-call fine-tuning at fixed retrieval counts. Measure valid formats, useful queries, and accuracy to identify how much of the open-backbone gap comes from protocol execution rather than visual reasoning.
• Dynamic contamination evaluation: Compare CR+ with VeriTaS and the event-novelty filter from Novel Claim or Déjà Vu? Probe parametric knowledge before measuring retrieval gains to establish actual need for external evidence, rather than assuming no contamination from publication dates.

17. Recent-work radar

1. VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking(ACL 2026): upgrades static snapshots to a dynamic benchmark and received an Oral presentation, directly addressing future pretraining contamination of CR+.
2. Novel Claim or Déjà Vu?(2026-07): challenges publication-after-cutoff as a guarantee of no contamination, examining event and semantic novelty. It is a necessary stress test of DEFAME's temporal claims.
3. RW‑Post / AgentFact(CVPRW 2026): combines real-world data containing original posts, reasoning, and evidence with five agents, offering a stronger setting for cross-auditing than multiple roles prompted within one MLLM.
4. T²Agent(AAAI 2026): combines tool use with Monte Carlo Tree Search, worth comparing with DEFAME's NEI loop under equal budgets.
5. ReMMD(2026-06): spans 500 samples, 2,756 images, five languages, and multi-image posts, addressing DEFAME's English single-image limitation, though still small.
6. Evidence Triangulation for Multimodal Fact-Checking in the Wild(2026-06): emphasizes triangulating real-world evidence, useful as a source-independence component rather than simply increasing search counts.
7. RAMA(2025-07): uses retrieval-augmented multi-agent systems for multimedia misinformation; role separation offers a candidate way to interrupt DEFAME's self-confirmation bias.

18. Conclusion: key lessons and limits

The reusable contribution is three principles rather than six stage names: treat images as evidence rather than caption attachments; let evidence gaps drive tool selection; and retain investigation traces with clickable sources. Table 4 and cost results show that dynamic planning primarily provideshigher accuracy with fewer queries, while task-matched retrieval tools provide the largest performance contribution: Web Search for MOCHEG, RIS for VERITE.

The quantitative conclusion has two parts. On temporally shifted CR+, DEFAME improves over direct GPT-4o by +34.5 pp, strongly supporting retrieval as a remedy for knowledge cutoffs. Yet it does not lead every subtask: VERITE T/OOC remains below AITR by 4.3 pp. More importantly, retrieved pages are not yet validated for source independence, credibility, and sentence-level entailment. The appropriate product role is thereforean evidence-gathering and drafting assistant for fact-checkers, rather than an autonomous arbiter of truth.

Files and index notes: This report is saved with identical content in HTML, Markdown, and plain text;ml-report-index.md was updated. The single-file HTML follows Warm Editorial Report conventions with inline CSS/JS, light/dark themes, horizontally scrollable mobile tables, and keyboard-accessible tooltips. The official validator returned 0 warning. Playwright was actually invoked, but /usr/bin/chromium、/opt/pw-browsers and its cache contain no Chromium executable, producing executable doesn't exist. As instructed, playwright installwas not run; completion of the three-view screenshot checks cannot be claimed.