What does the tangermeme toolkit reveal about cis-regulatory logic in deep learning models?

The gap opens after prediction accuracy

Genomic deep learning models now predict transcription factor binding, chromatin accessibility, histone modification, splicing and other modalities directly from nucleotide sequence. The paper's premise is that predictive performance is only the first step: biological discovery depends on what researchers do with a trained model afterwards, such as estimating variant effects, attributing predictions to nucleotides, probing global cis-regulatory rules or designing sequences with desired properties.

That downstream work is largely independent of architecture and training choices, yet no optimized repository generalized across model types existed. Analysis code tended to ship with individual models, while packages such as Captum implement one or a few related algorithms and others such as Selene, kipoi, CREsted, EUGENe and gReLU concentrate on training, fine-tuning and model zoos. Tangermeme is positioned deliberately as 'everything-but-the-model': it leaves loading and defining models to the user and implements the operations around them.

Stacking sequence edits with any model operation

The toolkit separates two kinds of building blocks. Sequence manipulations create inputs, and model operations consume them, including predictions, DeepLIFT/SHAP, in silico saturation mutagenesis, PISA or a custom operation. Because the two layers are independent, they can be stacked: in silico marginalization compares predictions before and after substituting a short sequence into a template, ablation compares predictions before and after altering a region the model may respond to, and variant effect estimation compares predictions before and after one or a few substitutions.

The paper illustrates this with simulations using ChromBPNet profile predictions on 100 randomly generated 2,114-bp sequences. A 'ATGATGTCAT' sequence was substituted into the middle of each sequence for marginalization, a window between 1,030 bp and 1,075 bp was dinucleotide shuffled for ablation, and a single nucleotide at position 1,053 was changed from G to A for variant effect prediction. These are simulated demonstrations of the API's composability, not measurements of biological function.

The same modularity supports DNA design. Screening generates random sequences, scores them with a user-defined objective and returns the best; greedy substitution iteratively tests all single-nucleotide variants and keeps the best at each stage; motif implantation inserts whole motifs from a database. A construct marginalization method merges marginalization and design by searching for short constructs that shift model predictions on average across many background sequences.

Speed gains and a silent failure mode in attribution

The paper reports concrete engineering results. Tangermeme one-hot encodes the entirety of hg38 chr1 in under 2 seconds, roughly three times faster than the next fastest implementation tested. Built-in batching means DeepLIFT/SHAP no longer requires all background sequences to sit in GPU memory at once, which the paper says removes that requirement without compromising exactness, and low-precision support further reduces the burden.

A second finding concerns correctness rather than speed. DeepLIFT/SHAP overrides the backward pass of nonlinear operations and identifies them through an internal dictionary; implementations can silently fail when they meet a nonlinear operation outside that dictionary or when activations are reused, and documentation or warnings are often absent. Users may notice only when convergence deltas, which should be near zero, become much higher. Tangermeme instead raises a warning when deltas are too high, can return the deltas for monitoring, and accepts custom nonlinear functions. In the paper's comparison, tangermeme correctly registered a custom profile head that Captum did not easily allow.

A seqlet defined by statistical significance

A seqlet is a contiguous span of nucleotides with high attribution values. Tangermeme's new caller defines one more strictly as a span whose attribution sum is statistically significant against a null distribution, with the additional requirement that all subspans within it would also be called seqlets by the same test. Subspans below a default minimum length of 4 are exempt, so noise does not break a call and seqlets may contain uninformative characters.

The algorithm bins attribution values with min-max scaling, builds a frequency histogram, and derives null distributions for each candidate seqlet length by asking what attribution sum random selection of that many values would produce. These distributions are converted to 1 minus cumulative distribution functions so P values become an indexing operation. The caller then scans all starting positions and lengths and decodes the longest non-overlapping spans passing the threshold. The paper states this makes it the first seqlet-calling algorithm able to call variable-length seqlets directly.

The parameters are few: the P value threshold, minimum and maximum seqlet lengths, optional flanks and the number of bins. The paper identifies the P value threshold as probably the most important to tune, notes that minimum length may matter when attributions are noisy, and reports that the number of bins is broadly robust and mostly affects compute time and memory.

Recursive caller versus TF-MoDISco on a known motif

The comparison used a synthetic task: 100 sequences of length 2,114 with uniform nucleotide probabilities, the MYC motif 'CACGTG' inserted at the center, and DeepLIFT/SHAP attributions from a MYC BPNet model. Seqlets called by the recursive algorithm were shorter than those from TF-MoDISco's approach, which uses a long fixed default length, and TF-MoDISco covered much more of each example even at a very permissive P value threshold.

Treating the central 6 bp as positive labels and seqlet calls as predictions, the paper swept P value thresholds and found precision and recall scores that TF-MoDISco falls near, suggesting underlying similarities between the approaches. The authors caution that this comparison is imperfect because a caller would also be correct if a MYC motif occurred elsewhere and drove predictions, but note that this penalizes all callers equally and does not prevent identifying broader trends.

Timing told a different story. TF-MoDISco was faster with very few sequences, but the recursive algorithm was faster at modest-to-large numbers of sequences, both in examples per second and seqlets per second. The paper explains the design difference: TF-MoDISco's wide and overcalled seqlets suit a larger pipeline that splits and quality-controls them, whereas the recursive caller is meant to be a stand-alone user-facing algorithm.

From seqlets to motif counts and spacing

Called seqlets are annotated by mapping their nucleotide sequence to a motif database with Tomtom-lite; in these evaluations the database was the human ChIP-seq slice of JASPAR2024, and each seqlet received the motif with the lowest P value. Annotations then become statistics: a count matrix of sequences by motifs, pairwise counts of motifs appearing in the same sequence, and spacing relationships.

Pairwise counting has modes that change what the number means. Symmetric counting records motifs A and B in both rows, yielding a symmetric matrix; asymmetric counting follows motif order from start to end without accounting for strandedness, which the paper says can be useful when order matters, such as for transcription initiation, but requires manually ensuring strandedness is correct. Symmetric is the default. These are different measurements of co-occurrence, not interchangeable summaries.

Two models for the same target learn different logic

The demonstration compares learned logic across models at the PLD6 promoter. Attributions showed that alternate sets of motifs drove predictions in different models, and BPNet and Beluga showed different motifs driving MYC predictions. After automatic calling and annotation, BPNet's predictions were solely driven by a MYC motif, whereas Beluga was driven by many motifs that also drive accessibility and transcription initiation.

Counting annotated seqlets across all MYC peaks and examining pairwise occurrences made the contrast visible at scale: BPNet focused almost exclusively on MYC motifs, while Beluga learned a broader lexicon of motifs and interactions also associated with chromatin accessibility. The paper's conclusion is that the learned logics are significantly different despite the models being ostensibly trained to predict the same phenomenon. This is a statement about what the models learned, not about which model is biologically correct.

Where the toolkit transfers and where it stops

The paper's own scope is transcription factor binding and chromatin accessibility models, but it states that tangermeme can be used with models predicting any modality, including alternative splicing, transcription, enhancer activity, and even RNA stability and binding. Operations support any internal computational unit, including convolutions, long short-term memory models, transformers and custom operations, and any prediction type from a single binary value to an entire profile. Sequence length is limited only by the memory needed to run one sequence through the model.

Two boundaries matter for interpretation. The seqlet comparison rests on a synthetic dataset with a single inserted motif and an imperfect labeling scheme, so the precision and recall trend describes caller behavior on that task rather than validated motif discovery in real genomes. And the BPNet-versus-Beluga contrast is a demonstration of what the pipeline surfaces, not an adjudication of the underlying biology.

The paper's forward-looking priority is documentation for large language model-based coding agents, so that such systems reuse tested, composable building blocks instead of recreating analyses and leave less code for researchers to audit. The practical distinction the paper draws is therefore between tools that train models and a toolkit that standardizes what happens after training, with the seqlet caller's variable-length, significance-defined calls as the specific capability most worth checking against an existing pipeline.

Sources used in this answer