What We Work On

Tiny Recursive Models

a 7M-parameter model that solves hard puzzles by revising its own answer

• Beats frontier LLMs on Sudoku — with <0.01% of their parameters (7M vs 100B+).
• Extremely data-efficient: ~1,000 training samples are enough to solve Maze-Hard.
• Replaces chain-of-thought with multi-level recursion — and reasons its way through ARC-AGI.

Jolicoeur-Martineau, Less is More: Recursive Reasoning with Tiny Networks, 2025.

TRM · the architecture

improve the latent state first, then improve the prediction

1

latent N_ZLprediction N_ZHouter N_sup

removeouter supervisionprediction refinementreset

Strip both loops and what is left is a plain 2-layer transformer — exactly the main baseline we compare against later.

2

3

4

1$x$ — the input question.

2$y$ — the current answer. Starts as noise.

3$z$ — the latent reasoning state.

4Putting it together — read $(x, y, z)$ → update $z$ a few times → use $z$ to improve $y$ → repeat.

how TRM really worksclick to play / pause

From puzzles to tissue

would recursion over space benefit spatial omics?

a cell is fixed by its row, column and box

a spot is shaped by its neighbours — DLPFC, cortical layers

expression is shaped by tissue architecture — HER2ST, breast tumour

Tasks we train/eval on

expression → domains, and histology → expression

Task 1 · SDI

Spatial domains · DLPFC

• Input: whole section — a sequence of per-spot PCA-transformed gene expression profiles
• Output: joint prediction of cortical class per spot (7 layers/classes)
• In effect a segmentation task: joint multi-class classification over all spots
• Dataset size: 12 human cortex sections
• Eval setup: leave-one-slice-out — test slice transformed into train PCA space
• Metric: ARI — higher is better

Task 2 · H2G

Histology → expression · HER2ST

H&E · 224×224 patch per spot

expression · 300 genes per spot · 35×35 grid

• Input: whole slide — a sequence of per-spot FM patch embeddings
• Output: joint prediction of expression of 300 genes across the whole slide
• In effect a regression task: log-normalized, continuous expression
• Dataset size: 36 breast-cancer slides
• Eval setup: held-out patient slide
• Metrics: PCC — higher is better · RVD — lower is better

Existing work for both tasks

how each task is usually solved

Our contributions

Baselines to keep recursion honest

same 2-layer block, same parameters — only the loop changes

control 1

No-Recursion 2-Layer

Same parameters and backbone architecture as TRM — equal to a single model forward pass.

control 2

Universal Transformer ×21

Depth-matched: 2 layers × 3 × (6+1) = 42 effective depth. No prediction refinement, no outer supervision.

control 3

336-layer unrolled net

2 × 3 × (6+1) × 8 (N_sup steps) = 336 layers. Fails to fit even the training data.

plus published per-slice methods: SpaGCN, STAGATE, GraphST, 3d-OT (SDI) · HisToGene, BLEEP, TRIPLEX, STEM (H2G)

latent N_ZLprediction N_ZHouter N_sup

removeouter supervisionprediction refinementreset

Strip both loops and what is left is a plain 2-layer transformer — exactly the main baseline we compare against later.

Domain identification, zero-shot

TRM beats both matched baselines on all 12 held-out slices

† per-sample unsupervised, trained on test slice · green: best · bold: ours

SDI: Prediction refinement over latent recursion steps

Held-out slice 151508. One latent step: noise (ARI 0.003). Three: the gross layers appear (0.417). Six: the boundaries sharpen (0.489).

SDI: Does it transfer to a different tissue?

train on brain, infer on breast (20 GT classes)

Method (zero-shot unless noted)	Max ARI
PCA + k-means	0.289
No Rec. 2-Layer	0.343
No Rec. 2-Layer + spatial	0.458
TRM latent	0.388
TRM latent + spatial	0.538
GraphST — fit on the slice	0.577
3d-OT — fit on the slice	0.683

We project breast data into the cortex PCA space before clustering. Reality check: adding coordinates helps the plain baseline too (+0.115) — the spatial gain overshadows recursion here.

The reasoning latent, learned on cortex, still organizes an unseen cancer tissue — close to methods trained on that very slice.

Histology → expression: best supervised result

B1 holdout, STEM protocol, 300 genes

Method	PCC-10	PCC-300	RVD
No Rec. 2-Layer	0.865	0.762	0.257
Univ. Transformer	0.830	0.745	1.088
TRM (ours)	0.883	0.785	0.354
HisToGene †	0.681	0.525	10.34
BLEEP †	0.773	0.565	0.603
TRIPLEX †	0.791	0.577	0.643
STEM †	0.830	0.598	0.069

STEM keeps the best RVD — it’s a diffusion model, built to match variance. We close most of that gap with adaptive halting, two slides from now.

Clustering TRM’s predicted expression recovers the pathologist’s annotations on the held-out slide.

How deep is deep enough?

which axis of recursion matters

SDI · (a) ARI saturates at ~3 blocks. (b) higher N_ZH lets more N_ZL steps help. (c) cutting N_ZL from 6 to 1 costs 0.25 ARI.

H2G · (a) PCC also saturates at ~3 blocks. (b) prediction refinement drives it: N_ZH 1→3 improves PCC 0.51→0.72. (c) in a complete forward pass, performance is already saturated and additional N_ZL adds nothing.

latent N_ZLprediction N_ZHouter N_sup

removeouter supervisionprediction refinementreset

Strip both loops and what is left is a plain 2-layer transformer — exactly the main baseline we compare against later.

Adaptive computation time (ACT)

let the model decide when to stop recursing

Test-time scaling

The model automatically allocates compute for each problem at inference.

ACT

Lets the model determine computation through the number of reasoning steps — recursion, for us.

1Each outer recursion (N_outer) updates $(z, y)$ and a halt head predicts $q_k = \sigma(\mathrm{halt}(z_k))$.

2Inference: stop at the first $k$ with $q_k > \theta$; otherwise run to $K_{\max}$.

3Training: $\mathcal{L} = \sum_k \big[\mathcal{L}_{\mathrm{task}}(y_k) + \mathrm{BCE}(q_k, t_k)\big]$, where $t_k$ indicates whether the answer is good enough (binary: correct/incorrect in discrete tasks).

4Discrete tasks (Sudoku): $t_k = \mathbf{1}[y_k = y^*]$. Regression has no exact match — $t_k$ must be built.

click for ACT

click to return to the full figure

Halt recipes for H2G

what to supervise $t_k$ with, on a held-out subset of each slide’s spots

Halt step, 36 slides: bimodal at 2 and 8; the unseen test slide halts at 5, between the groups.

The 0.4 threshold cleanly splits the halt-probability distribution into early-halt vs never-halt slides.

Random-K training (K ~ U[2,16]) restores test-time scaling — PCC 0.69 → 0.72 from N_outer 1 → 4 — the gradient ACT needs.

Does reasoning help?

What’s next

recursion at whole-slide scale — and beyond

Whole-slide Xenium · H&E → expression, zero-shot

Ovary · platform transfer · top-50 PCC 0.636 vs 0.145 (Phoenix, 1.2B) · ours: 2.1M params

Cervix · held-out organ · top-50 PCC 0.428 vs 0.143 (Phoenix)

Also in progress

Perturbation predictionDrug and CRISPR response with the same recursive operator.

Recursive flow matchingFlow matching with recursion over local tissue neighbourhoods.

Equilibrium reasonersRecursion as convergence to a fixed point.

When does looping a small model beat scaling a big one?

What is the right stop signal for continuous outputs? · If the encoder is a foundation model, is the latent state ever needed?

code · github.com/AI4SCR/recursive-reasoning-omics · dhruva.rajwade@chuv.ch