Official PyTorch implementation of Supply And Demand Loss (SADL), with code to train and evaluate LeWM and INTACT world models.
Junhan Sun1
· Yigui Feng2
· Mingzhe Liu2
· Haoyan Xu2
· Lei Ye2
· Ziqi Han2
· Hanru Bai3
· Yanzhao Zhang3
Hao Zhao2,4,†
· Guofeng Zhang1,5,†
1State Key Laboratory of CAD&CG, Zhejiang University · 2RoboParty Lab · 3Alibaba Group
4AIR, Tsinghua University · 5InSpatio · †Corresponding authors
Quick start · Why SADL? · Method · Results · Citation
- [2026-11-19] Released the project page and project video.
- [2026-11-16] Released the paper on arXiv.
- Released the INTACT and LeWM training and evaluation code and the Hugging Face repository.
Watch the 3-minute project video · Open the interactive project page
Start with Python 3.10 and a CUDA-enabled PyTorch installation.
git clone https://github.com/zju3dv/SADL.git
cd SADL
python -m pip install -r requirements.txtFor the optional fast encoder kernels on Hopper or Blackwell GPUs, run python -m pip install -r requirements-fast.txt.
This example runs the four-task INTACT model on Reacher using the Direct planner. It downloads the Reacher checkpoint and dataset, then runs a 10-episode check on one GPU.
hf download DavidSunok/SADL --include "checkpoints/intact/four_task/seed3072/reacher/*" --local-dir hf
hf download DavidSunok/SADL datasets/reacher.h5 --local-dir hf
python evaluate.py --checkpoint hf/checkpoints/intact/four_task/seed3072/reacher \
--task reacher --data-dir hf/datasets --planner direct --seeds 0 --episodes 10Omit --seeds 0 --episodes 10 to run the published evaluation protocol of 100 episodes for each of seeds 0, 1, and 42.
On single-node RTX 5090 or 5090 D systems that report illegal memory access at the first NCCL collective, run export NCCL_SAFE=1 before either training command below.
The training entry point uses four GPUs and a dataset manifest covering all four tasks. Download the HDF5 files once, then choose either training command.
hf download DavidSunok/SADL --include "datasets/*.h5" --local-dir hf
python tools/prepare_datasets.py --root hf/datasets --output datasets.json
# INTACT trained jointly on PushT, Cube, Reacher, and TwoRoom
python train.py --cell INTACT_4T_S3072 --gpus 0,1,2,3 --datasets datasets.json
# LeWM trained on PushT
python train.py --cell LEWM_PUSHT_S3072 --gpus 0,1,2,3 --datasets datasets.jsonAfter training, pass the exported task directory to the evaluator.
python evaluate.py --checkpoint runs/INTACT_4T_S3072/exports/reacher \
--task reacher --data-dir hf/datasets --planner directSee the training guide for all model and task combinations, hyperparameters, output files, and how to resume training.
In Joint-Embedding Predictive Architectures, a well-regularized projector output does not by itself prevent representation collapse.
- From initialization, the encoder representation
$H$ is dominated by a component shared by all inputs: sample variation carries about 0.01% of its energy. - A batch-normalized projector subtracts the batch mean. Moving all of
$H$ along that shared direction leaves the endpoint$Z$ unchanged, so the Demand Regularizer and the task loss, both computed from$Z$ , get zero gradient there. - The remaining differences shrink until the representation collapses, while
$Z$ can still look well-constrained.
SADL keeps the original objective and adds a one-sided Supply Regularizer on the encoder. It introduces no auxiliary networks, no trainable parameters and no architecture changes, and it works with any Demand Regularizer.
50% → 0% collapse rate over ten seeds · 3.4–6.4× encoder effective rank at epoch 1 · >95% search-free control
SADL reads a detached demand capacity
Training and evaluation code for JEPA world models trained on one or more control tasks with one BatchNorm population that does not depend on the GPU or the task, for two objectives:
- INTACT: latent prediction and goal-conditioned action, evaluated with Direct and Guarded A planning;
- LeWM: latent prediction only, evaluated with CEM planning.
Both objectives use SIGReg on the projected embeddings and the SADL regularizer between encoder and projector embeddings. Tasks: PushT, Cube, Reacher, TwoRoom.
- Every optimizer step uses a global batch of 256 windows of 8 frames. Windows are assigned to the BatchNorm population by their position in the global batch before they are split across GPUs; the projector and the prediction projector use one synchronized BatchNorm population over that batch. Multi-task batches are task-balanced (64 windows per task) and shuffled, so every normalization call sees all tasks.
- At inference all tasks use the same saved running statistics: no task-specific normalization, no statistics re-estimation and no task input. The task only selects the action dimension and the action scaling.
- One action encoder, predictor, prediction projector and (INTACT) actor are shared by all tasks.
train.py train one cell (4 GPUs)
evaluate.py evaluate an exported checkpoint
docs/TRAINING.md single-task and four-task training guide
tools/ dataset list
src/ training code; src/model: model, data, planners and configs; src/kernels: GPU operators
eval/ evaluation protocol: episode sampling, causal action history, planner settings
--planner direct or guarded_a for INTACT checkpoints, cem for LeWM checkpoints. Start states and goals are
sampled from dataset episodes with the goal 25 steps ahead; the agent has at most 50 actions, plans 5 action blocks
ahead and replans every 5 actions; INTACT receives the causal history of the previous 5 actions; the model runs in
FP32 with solver batch size 1. By default evaluate.py runs seeds 0, 1 and 42 with 100 episodes each. Reported
results were computed on an NVIDIA RTX 4090.
| Planner | Candidates |
|---|---|
direct |
none: the actor's action plan |
guarded_a |
128 samples x 3 iterations around the Direct plan, top 16, initial std 0.25 |
cem |
300 samples x 30 iterations, top 30, initial std 1.0 |
Checkpoints for the shared BatchNorm results and the four HDF5 datasets are available from DavidSunok/SADL on Hugging Face under
checkpoints/<intact|lewm>/<four_task|single_task>/seed<seed>/<task>/ (weights_epoch_1.pt, config.yaml,
action_stats.json, training_config.json) and datasets/. Pass a task folder to evaluate.py as --checkpoint
and the dataset folder as --data-dir. The four task folders of a four-task model share all weights and
normalization statistics.
Gaussian or not, the encoder keeps up with what is demanded: with SADL, LeWM reaches 62.8–75.3% after one epoch under all eight Demand Regularizers (from 23.8–47.7%), and INTACT reaches 95.4–95.6% search-free success (from 28.2–54.4%). Mean over three training seeds.
SADL can also reopen an encoder that has already collapsed. Resuming from fixed near-collapsed checkpoints with SADL at its default weight
SADL is a drop-in addition to standard self-supervised objectives. On ImageNet-100 with ViT-S/16, under a learning rate ×4 and no warmup, it lifts VICReg by 8.4 points and BYOL by 4.9 points of online linear accuracy, with no other change to the recipe.
On ImageNet-LT, Galaxy10, CIFAR-10 and Imagenette, SADL improves early learning; on Imagenette it reaches the baseline's final accuracy with 23% fewer updates. Interactive curves are on the project page.
Success rate (%), mean +- SD over training seeds 3072 / 3073 / 3074 (each seed: 300 episodes).
| Model | Training | Planner | PushT | Cube | Reacher | TwoRoom | Mean |
|---|---|---|---|---|---|---|---|
| INTACT | four tasks | Direct | 85.7 +- 0.9 | 100.0 +- 0.0 | 93.4 +- 1.7 | 100.0 +- 0.0 | 94.8 +- 0.4 |
| INTACT | four tasks | Guarded A | 88.1 +- 0.8 | 99.6 +- 0.4 | 93.1 +- 0.7 | 99.8 +- 0.4 | 95.1 +- 0.3 |
| INTACT | single task | Direct | 85.1 +- 0.4 | 100.0 +- 0.0 | 94.9 +- 2.0 | 99.7 +- 0.6 | 94.9 +- 0.5 |
| INTACT | single task | Guarded A | 89.8 +- 0.2 | 99.6 +- 0.2 | 93.7 +- 3.2 | 99.3 +- 0.9 | 95.6 +- 0.9 |
| LeWM | four tasks | CEM | 64.7 +- 18.5 | 59.6 +- 1.3 | 46.7 +- 26.1 | 94.4 +- 1.0 | 66.3 +- 9.2 |
| LeWM | single task | CEM | 83.1 +- 1.6 | 58.9 +- 1.4 | 81.0 +- 4.3 | 91.3 +- 1.5 | 78.6 +- 1.9 |
Per-seed values are in RESULTS.md.
MIT, see LICENSE. Parts of src/model derive from LeWM; see NOTICE.md.
@article{sun2026sadl,
title = {{SADL}: On the Demand-Supply Geometry of Representation Collapse},
author = {Sun, Junhan and Feng, Yigui and Liu, Mingzhe and Xu, Haoyan and
Ye, Lei and Han, Ziqi and Bai, Hanru and Zhang, Yanzhao and
Zhao, Hao and Zhang, Guofeng},
journal = {arXiv preprint arXiv:2611.21596},
year = {2026}
}