Modeling gene regulation from DNA sequence

Every cell contains essentially the same genome, yet cells use genes at different levels and at different times. Part of this regulatory program is encoded in DNA through promoters, transcription-factor binding sites, and combinations of elements whose effects depend on sequence context.

I study sequence-to-function models because they provide a direct way to connect DNA with molecular phenotypes such as binding, chromatin accessibility, and RNA abundance. Models including Enformer (Avsec et al., 2021) and Borzoi (Linder et al., 2025) show that long sequence context can support detailed predictions of regulatory activity. I am interested not only in prediction, but also in whether these models can identify consequential bases, anticipate variant effects, and transfer what they learn across biological settings.

Questions that drive my work

The first question is generalization. A genomic model can fit abundant measurements while relying on correlations that do not transfer to unseen genes, conditions, or species. Evaluation should make memorization difficult and test whether the model learned reusable biology.

The second is where prior knowledge should come from. Functional labels are expensive and concentrated in a few organisms, while genome sequence is abundant. Evolution leaves patterns of conservation and divergence that self-supervised models can learn. Species-aware language models show that evolutionary context can expose regulatory structure (Karollus et al., 2024), but the best pretraining scope depends on the target organism and task. A larger corpus is not automatically better if distant genomes dilute the relevant signal.

The third is biological interpretation. Attribution and in silico mutagenesis can highlight important bases, but a score is not yet a mechanism. I look for agreement with known motifs, controlled perturbations, independent variant measurements, and condition-specific responses. A useful model should produce hypotheses that can be tested rather than explanations that must be accepted on faith.

Directions I have explored

Learning regulatory priors from related genomes

My main project in this direction is Shorkie (Chao et al., 2025), a fungal DNA language model for budding yeast. We first trained a supervised sequence-to-expression model directly. In our experiments, it learned the genes it had seen but did not generalize as strongly as we wanted to unseen genes. This led us to test whether self-supervised pretraining on related fungal genomes could provide a better starting point.

We trained the same masked-DNA model across nested evolutionary scopes: one Saccharomyces cerevisiae reference genome, 80 strains, 165 genomes from the Saccharomycetales order, and 1,341 genomes spanning the fungal kingdom. Pretraining helped, but the largest corpus was not the best. The strongest transfer came from the 165-genome order, which supplied useful diversity while staying close enough to budding yeast for the learned patterns to carry over.

We then fine-tuned the model on high-resolution regulatory measurements, including time-course RNA-seq after transcription-factor induction. This connects static sequence with a dynamic response. Motif sensitivity and regulatory-variant scoring provide additional tests of whether pretraining contributed biological information rather than only improving optimization.

The published Shorkie and Shorkie_LM checkpoints are runnable in-browser on Shorkie Lab, where visitors can compute live feature attributions, layer tracebacks, occlusion maps, and zero-shot sequence constraint across curated yeast loci.

Connecting observation with intervention

Sequence models learn from naturally occurring variation and observational assays. Perturbational data ask a different question: what happens when a gene or pathway is deliberately changed.

I contributed to a genome-wide Perturb-seq effort at that scale (You et al., 2026), which reads the expression response to single-gene perturbations across roughly 100 million cells. Those measurements are the intervention side of the problem Shorkie approaches from sequence, and each is most informative read against the other.

Future directions

Evolutionary pretraining, natural variation, functional assays, and controlled perturbations are four windows onto the same regulatory program, and they are usually reported separately. Putting them in one evaluation framework is worth doing mostly for the disagreements: a model that predicts an effect the perturbation does not show, or a conserved element that no assay marks, is pointing either at a flaw in the model or at an experiment worth running. Telling those two cases apart is what would move a sequence model from a good predictor to a source of testable biology.