Two models, in the browser
Shorkie Lab
Shorkie predicts what a yeast sequence expresses. Shorkie_LM predicts what belongs at each base. Same encoder, different question — and the same drawing means opposite things in each.
Both pages run the real published models — the checkpoints from Chao et al. 2025, converted and executed in your browser, not re-creations or heuristics. Both work over the same 23 S. cerevisiae windows with the same curated annotation, so a result on one page can be read directly against the other — and the language model's constraint score is precomputed for the whole genome, so the third page can go anywhere those 23 do not.
Shorkie
“How much does this base change the prediction?”
The fine-tuned model: 16,384 bp in, coverage over 896 bins × 5,215 tracks out. Five attribution methods on one axis, a region traced back through every layer, an occlusion map of what drives what, and every curated binding site knocked out and measured.
Open the expression playground →language modelShorkie_LM
“How constrained is this base?”
The masked DNA language model Shorkie is built on, pretrained on 165 Saccharomycetales genomes. A four-way distribution at every one of the 16,384 positions: mask a base and ask what belongs there, read constraint as information content, and test it against what the annotation says is really there.
Open the language-model playground →genome browserThe whole genome
“Where in yeast is the model most certain?”
The same constraint score as the language-model page, precomputed for all 12,157,105 bases of the R64 reference and served as a tile pyramid: whole chromosomes down to individual letters, with 6,613 gene models alongside. Assembled from overlapping windows, keeping only each window's uncontaminated core — and every window on one pooling-grid phase, so any two positions in the genome are comparable.
Open the genome browser →interpretabilityAttention & Receptive Field Studio
“How does an enhancer 10 kb upstream communicate with a TSS?”
Layer-by-layer receptive field expansion from Conv Stem (11 bp) to Transformer Bottleneck (16,384 bp). Dual-probe enhancer-to-TSS dependency tracer, authentic 128×128 attention rollout heatmaps, empirical context convergence curves, and cross-architecture comparison (Transformers vs. Dilated CNNs vs. SSM/Mamba).
Open the attention studio →The same trunk, a different decoder
The two models share their entire encoder — an 11 bp convolutional stem, seven residual blocks that pool 16,384 positions down to 128, and eight transformer layers at that bottleneck. They diverge only in what they do on the way back up, and the divergence is not arbitrary: each decoder resolves exactly as finely as its output is defined.
| Shorkie | Shorkie_LM | |
|---|---|---|
| stem · residual tower · transformer | identical — the shared trunk | |
| U-Net decoder blocks | 3 (128 → 1,024, cropped to 896) | 7 (128 → 16,384, no crop) |
| head | 5,215 units, softplus | 4 units, softmax |
| output | 896 bins × 5,215 tracks | 16,384 positions × 4 bases |
| resolution | 16 bp — a coverage bin | 1 bp — a base |
| training | fine-tuned, lr 2e-5, 8 folds | masked LM, 15% mask rate, 1 fold |
Shorkie stops at 16 bp because coverage is a 16 bp quantity; the LM upsamples all the way back because a base is what it predicts. Everything else in the two networks is the same weights-shaped object trained toward a different loss.
Which page answers which question
- “Which bases drive this gene's expression?”
- Shorkie. Attribution is a statement about a prediction, and only the expression model makes one.
- “Is this stretch of sequence under constraint?”
- Shorkie_LM, if it is one of the 23 primary regions — that page has the full four-way distribution at every position. Low entropy in it means the model finds the base highly determined by its context: the model's own analogue of conservation, computed without an alignment.
- “…and if it is somewhere else in the genome?”
- The genome browser. It carries the same score for every base of R64, but only the score — one number a position, not which base the model expected. That is the trade: the four-way distribution genome-wide would be 49 M values, so it stays on the 23 regions and the browser covers everywhere else.
- “Is that peak a real binding site?”
- Either — both pages carry the same curated annotation layer from SGD, the Harbison/MacIsaac regulatory code, ORegAnno and JASPAR, drawn with its evidence tier.
- “What did pretraining buy?”
- Neither, yet. That needs the published
Shorkie_Random_Initablation, which is not on the site.
Connected resources
- 📝 Blog post — deep dive into the Shorkie architecture, evolutionary pretraining, and motif analysis
- 📄 bioRxiv preprint — Chao et al. 2025
- 🔬 Research overview — modeling gene regulation from DNA sequence
- 🧰 Software tools — all open-source research software
- 📖 Documentation site
- 💻 GitHub repository
- 🎬 ProbGen talk — conference presentation video
- 📊 Talk slides (PDF)
Source, weights and documentation: the Shorkie repository and its documentation site. The paper is Chao et al. 2025.