What Do Current Systematic Generalization Tasks Miss?
A Reasoning-Centered Analysis
TranSGrid is a grid-transformation task for studying systematic generalization. Given an initial and a target board, a model predicts a sequence of row, column, and 2 × 2 block operations that transforms one into the other. Browse the ten atomic actions below, then explore examples and replay the selected model's prediction step by step (Note: This demo shows precomputed evaluation results; it does not run models live).
Atomic actions
Every solution is a sequence of atomic actions. There are ten of them: four act on rows and four on columns (each either shifts one row or column by a cell, or swaps it with its neighbour), and two rotate a 2 × 2 block. Each card below applies one action, with example arguments, to the same 6 × 6 board. Rows and columns wrap around the edges of the board; a block must fit entirely inside it. Press Play to watch an action move the cells.
- Cells the action moves; arrows show where each one goes
- Cells whose digit changed
Interactive demo
Given to the model
Model prediction
Select an example
The current board will replay the selected prediction.