🧬 Evo 2: How a Unified Biological Foundation Model Redefines Genome Design

I. Research background

Problem and importance

The core bioinformatics challenge is: How can DNA sequences reveal function, and how can new sequences with desired functions be designed inversely? Traditional approaches use rule/domain-knowledge symbolic systems such as position-weight matrices or task-specific supervised models, both with limited generalization. Challenges include:

• Cross-species prediction: Can a human-trained model reliably predict equivalent functional elements in bacteria?
• Long-range dependencies: Short windows cannot capture DNA 3D structure, transcription-factor binding competition, or distant regulatory interactions.
• Generating new sequences: Beyond predicting mutation effects, create previously nonexistent functional sequences satisfying constraints.
• Unsupervised applications: Zero/few-shot inference on entirely unlabeled new tasks.

Foundation models become important by self-supervised pretraining on massive heterogeneous biological sequences to learn deep biological statistical regularities.

Standard settings and evaluation

Current AI4Science, particularly computational biology, commonly uses:

• Main data sources: NCBI GenBank, IMG/VR viral genomes, ENCODE human regulatory regions, GTEx tissue-specific expression.
• Task framework:
  • Sequence classification: promoters, enhancers, transcription-factor binding sites
  • Variant-effect prediction: pathogenic versus common SNPs, QTL prediction
  • Sequence generation/design: de novo genomes and artificial DNA with specified functions
  • Expression prediction: infer tissue-specific gene expression from sequence
• Benchmarks: ClinVar clinical pathogenicity labels, BRCA1/BRCA2 saturation-mutagenesis maps from in-vitro experiments, Deep Mutational Scanning databases.
• Metrics: AUROC/AUPRC for variants, classification accuracy, Pfam hit rate for generation, biological-consistency measures.

Major frontier disagreements

Several technical routes compete:

• Architectural choices: Transformer attention versus convolutional hybrids such as Mamba/StripedHyena. Hybrids claim better long-sequence efficiency and are used by Evo 2; attention offers greater interpretability.
• Training-data scope: Focus on high-quality human genomes (NT-v2) or include all domains of life (Evo 2)? The former may favor human tasks; the latter pursues scale/diversity but introduces noise.
• Modality unification: Learn shared DNA/RNA/protein representations in one model, as Evo 2 attempts, or use specialists such as ESM for proteins? Unification enables transfer but may dilute individual modalities.
• Encoding biological constraints: Let dynamics emerge from data, explored through SAEs, or inject explicit physical/biological rules? Evo 2 chooses emergence.

II. Core paper deep dive

Abstract and central question

Evo 2, an Arc Institute effort published in Nature in 2026, seeks a general biological foundation model across all domains of life.

Core question: Can one model represent and generate bacterial, archaeal, eukaryotic, and viral DNA while reaching or approaching SOTA in variant prediction, annotation, and generation?

In my words, Evo 2 demonstrates a 4-billion-parameter hybrid convolution–attention model pretrained in two stages on 9 trillion DNA base pairs, capable of:

• Zero-shot prediction of functional effects of SNPs, insertions, and deletions
• Generating biologically plausible genomes hundreds of kb long
• Inference-guided design of DNA with specified properties such as open chromatin regions
• Using sparse autoencoders to reveal learned transcription-factor binding sites, exon–intron boundaries, and other biological features

Motivation and observed limitations

Why Evo 2? Evo 1 (Science, 2024) succeeded but retained key limits:

• Limited biodiversity: Evo 1 mainly trained on prokaryotes, limiting eukaryotic, especially mammalian, generalization. Eukaryotic introns, enhancers, and heterochromatin differ substantially.
• Context bottleneck: Regulatory understanding requires long distances; enhancers may be tens of thousands of bases from promoters. Evo 1's context was limited.
• Unknown universality across life forms: Can all-life training learn genuinely species-independent biological logic?
• Insufficient variant edge-case coverage: Existing models degrade on non-SNV insertions, deletions, and complex rearrangements, despite their clinical importance.

Technical methodology

Data and pretraining stages

OpenGenome2 dataset: 8.8 trillion nucleotides collected from:

• Bacteria and archaea via GTDB
• Eukaryotic plants, fungi, and animals via NCBI
• Viruses via IMG/VR, excluding known human pathogens such as influenza/HIV for biosafety

Deduplication is crucial: raw GenBank is highly biased, with some strains sequenced hundreds of times. Homology-based clustering ensures novelty up to a similarity threshold.

Two-stage pretraining:

1. Short-context pretraining(8192 tokens): Focus on gene-rich regions to quickly learn local coding/promoter patterns. Shorter contexts improve convergence and sample efficiency.
2. Multistage mid-training(progressively to one million tokens): Longer context teaches long-range regulation. Expensive, but needed for chromosome-scale properties.

Architecture: StripedHyena 2

The core uses StripedHyena 2, a convolutional multi-hybrid combining three operations:

• Input-dependent convolution: Unlike fixed-filter convolution, kernels adapt to inputs, learning input-specific local features.
• Selective mechanisms: Borrowing from Mamba-style state-space models, selectively attend to positions when needed, similar to attention but more efficient.
• Hybrid layers: Alternate convolution and attention through depth to balance efficiency and expressivity.

Why not pure Transformers? At one million tokens, quadratic complexity explodes memory/compute. StripedHyena 2 claims 3× speedup with perplexity and downstream performance on the same data comparable or better.

Loss and training objective

Standard causal next-token prediction:

This established autoregressive objective is not Evo 2's innovation; rather, careful data selection and multistage scheduling are. It also uses dynamic batching by sequence length to maintain GPU utilization.

Downstream inference configuration

A key choice is adapting pretrained language models to classification and generation:

• Variant-effect prediction: Embed reference and variant inputs, use their difference as features for a lightweight supervised classifier, avoiding full 40B-model fine-tuning.
• Sequence generation: Autoregressive sampling, optionally constrained by functional properties. Inference guidance uses joint probabilities:
  P(x|c) ∝ P(x) · P(c|x)^α
  where c is a constraint such as chromatin accessibility and α a temperature.

Experimental design and results

Benchmark 1: Variant effects

Key finding: For non-SNVs, Evo 2 is the only reliably functioning unsupervised model. This matters clinically because many pathogenic variants are insertions/deletions rather than point mutations.

Benchmark 2: Gene annotation

Tests identifying exon boundaries from DNA alone, without protein information. Consistency evaluation on 8 held-out species:

• Classifier using Evo 2 embeddings: AUROC 0.91-0.99
• Outperforms AUGUSTUS and SegmentNT
• Especially strong cross-species generalization suggests species-independent gene-structure learning

Benchmark 3: Sequence generation

Prokaryotic generation (M. genitalium, 580 kb):

• Among generated sequences, 70% of genes have significant Pfam homology matches, suggesting genuine protein domains
• Versus Evo 1's 18%, a 3.9× improvement
• suggesting more structured genetic code rather than random sequences

Eukaryotic generation(yeast, S. cerevisiae, 330 kb):

• Generated protein-length distributions closely resemble natural ones
• Successfully generates complex eukaryotic structures with multiple introns/exons

Benchmark 4: Chromatin accessibility design

The most innovative application specifies which DNA regions should bind histones and which should remain open and designs sequences satisfying those constraints.

• ATAC-seq validates generated open-chromatin patterns in HEK293T human kidney and K562 leukemia cells
• Experimental success exceeds random controls, suggesting nontrivial cell-type-specific regulatory logic
• This is A rare example spanning inference guidance directly to in-vitro validation, strongly supporting biological meaning in learned representations

Key innovations and insights

• Architectural innovation: Demonstrates efficient million-token hybrid convolution–attention without sacrificing expressivity.
• Data strategy: Nine trillion bases across life domains, careful deduplication, and staged scheduling are central.
• Interpretability advance: SAEs disentangle discrete transcription-factor and exon–intron features from a 40B model, achieving mechanistic interpretation at this genomic scale for the first time.
• Cross-modality reach: Unified DNA, RNA, and protein patterns, though DNA dominates the paper, pioneer general biological-sequence representations.

Narrative summary: From task-specific small models before AI4Science, to single-species foundations (Evo 1), to unified all-life models (Evo 2): biological scaling laws work through careful data/architecture choices addressing biodiversity.

III. Related work and technical evolution

Overview

Genomic foundation models have three clear stages:

Stage 1: Early specialist models (2018–2021)

Representatives: DNA-BERT, DNABERT-2 (BERT variants for DNA)

• Approach: Directly transfer masked language modeling from NLP to DNA. Small ~100M-parameter models train on human-genome subsets or single species.
• Advantages: No labels required; fast training on moderately sized GPUs.
• Limitations:
  • Short 512–1024-token context cannot capture distant regulation
  • Substantial degradation on new species/tasks
  • No generation capability, only classification/embeddings

Stage 2: Scaling and multiple species (2022–2024)

Representatives: Nucleotide Transformer (Nature Methods 2024), Evo 1 (Science 2024)

Nucleotide Transformer:

• Innovation: Trains on genomes from 850 species with tens of billions of base pairs. The key insight is that multispecies pretraining improves generalization.
• Result: SOTA on human-genome tasks and substantially better cross-species transfer than DNA-BERT.
• Limitations: Still human-centered, biased toward humans/common model organisms, with incomplete eukaryotic-regulation learning.

Evo 1 (a breakthrough in scientific sequence modeling):

• Paradigm shift: First demonstrates that prokaryote-specific models can reach unprecedented 7B–40B scale and performance, far exceeding SOTA on prokaryote-related benchmarks such as ClinVar.
• Technical choices: Uses original Transformers with ~4k–16k context, sufficient for prokaryotic properties.
• Unresolved questions:
  • Can it extend to eukaryotes?
  • Is million-token context really necessary?
  • Are Transformers optimal at such lengths?

Stage 3: Unified, efficient, interpretable (2025–2026)

Evo 2 and contemporaries:

• Evo 2: All life domains, StripedHyena 2 hybrids, 1M-token context.
• Parallel developments:
  • HyenaDNA(2024): Pioneering evidence for convolutional advantages on long sequences
  • Caduceus-Ph(2025): Bidirectional state-space models competitive on some tasks
  • Biological sparse-autoencoder applications(2025–2026): Interpretability advances through decomposing biological features from large models

Evolution and resolution of key questions

Question 1: Does multispecies learning dilute specificity?

• Nucleotide Transformer's answer: “No; diversity increases learning signal.”
• Evo 1's further confirmation: “With more data and larger models, multispecies and even multimodal learning improve.”
• Evo 2's conclusion: “Even with viruses and archaea, proper deduplication lets all-life learning retain or improve performance—a scaling-law success.”

Question 2: Is long context necessary and worth the cost?

• Before Evo: 4–16k tokens seemed sufficient.
• Evo 2 evidence: At one million tokens, models learn distant regulation, genomic structure, even evolutionary relationships. Efficient StripedHyena 2 architecture is needed to amortize the cost.
• Practical implication: Long-context value extends beyond limited performance gains to richer representations and better generation.

Question 3: Where is the architectural ceiling?

• Criticism of Transformers: O(n²) becomes an absolute bottleneck at one million tokens.
• Convolution and & state-space revival: Mamba/StripedHyena show O(n) or O(n log n) architectures can achieve perplexity comparable to or better than Transformers while running much faster.
• Evo 2's position: Convolution plus lightweight attention appears a sweet spot between efficiency and expressivity.

Remaining questions and next directions

Most valuable next research directions(ranked by expected benefit / difficulty):

1. Interpreting cross-modal fusion(high benefit, moderate difficulty): Use SAEs to describe DNA/RNA/protein interactions precisely, guiding multimodal pretraining design.
2. Systematic inference-guidance optimization(moderate-high benefit, moderate difficulty): Current guidance is prototypical; better constraint encoding/sampling could improve design practicality.
3. Zero-shot microbial-function prediction(moderate benefit, low difficulty): Apply Evo 2 directly to unlabeled environmental microbes once data are available.

IV. Structured review and independent analysis

This central section examines Evo 2's value and pitfalls.