Leave. Come back. Same world. And you can edit it.
Video world models forget. Turn the camera away from a chair for a few seconds and turn back, and it is a different chair, somewhere else, or gone. The standard fixes give the model more implicit memory — a longer context window, or a compressed latent state. Both decay with how long you looked away, because both store the world as activations.
Cairn takes the world out of the weights. Objects live in an explicit, external ledger — persistent id, pose, appearance, provenance — written by perception running on the model's own generated frames, and read back to coerce generation when the camera returns. Looking something up in a table costs the same whether you looked away for 4 frames or 400.
Two consequences, both measured below:
- Return consistency stops depending on absence duration. Every baseline collapses to 0% by 128 frames away. Cairn holds 100%, with a flat 3.6 cm position error.
- The world becomes editable.
ledger.move(obj, pose),ledger.remove(obj),ledger.set_attr(obj, k, v)— including for objects that are off screen when the command is issued. The baselines cannot express the operation at all.
from diffusers import LTXPipeline
from cairn.pipelines import CairnPipeline
pipe = LTXPipeline.from_pretrained("Lightricks/LTX-Video")
cairn = CairnPipeline.from_pipeline(pipe) # ← the one line
clip = cairn("a living room", camera=path[0:24]) # generate, then commit what was drawn
cairn.edit.move(3, position=[1.2, 0.0, 0.4]) # edit the world, off screen or not
clip = cairn("a living room", camera=path[24:48]) # the return obeys the ledgerNo GPU? The same API runs on a surrogate generator that reproduces how autoregressive video drifts:
cairn = CairnPipeline.surrogate(scene=make_scene(8, seed=0))pip install -e ".[bench]"
cairn demo --absence 64 # A/B/C/D on one leave-and-return, side by side
python benchmarks/run.py # the full sweep, ~3 min on a laptop CPU
pytest # 114 tests, ~25 sOr try it with nothing installed at all: the Space runs the same library in your browser via Pyodide.
flowchart LR
subgraph GEN["Video generator (any diffusers video pipeline)"]
S["sampler<br/><i>implicit state — drifts</i>"]
end
subgraph CAIRN["Cairn"]
direction TB
L[("<b>WorldLedger</b><br/>persistent id · pose · appearance<br/>provenance · transaction log")]
C["<b>CommitPath</b><br/>segment → measure → re-identify<br/>merge or register"]
R["<b>RetrievePath</b><br/>i. appearance cross-attention<br/>ii. anchor latent inpaint<br/>iii. verify & resample"]
D["<b>ConsistencyDebt</b><br/>scalar divergence<br/>threshold → correct"]
E["<b>EditAPI</b><br/>move · remove · set_attr<br/>undo · rewind"]
end
S -- "generated frames" --> C
C -- "rows (earned, never oracular)" --> L
L -- "what should be on screen now" --> R
R -- "ForcingSpec" --> S
C -- "detections" --> D
L -- "expectation" --> D
D -- "escalate strength, redraw" --> R
E -- "transactions" --> L
style L fill:#2f7ab8,color:#fff
style CAIRN fill:#f4f7fa,stroke:#2f7ab8
The ledger is deliberately boring — a dict of rows plus an append-only log. Its power is three properties no learned latent has:
| property | consequence |
|---|---|
| exact | a stored pose does not drift while unobserved, so return error is O(1) in absence duration rather than O(√t) |
| addressable | ledger[7] names one object, so move(7, …) is well-defined. A compressed latent has no row 7 |
| auditable | every mutation is an invertible transaction, so the world rewinds to any prior instant |
All four run the same generator, the same drift dynamics, the same seed, the same
perception front end, and the same audit ledger. The only line that differs is the memory
policy — runner.make_policy. A test asserts this
(test_only_the_memory_policy_differs_between_conditions).
| policy | why it eventually fails | |
|---|---|---|
| A | Vanilla autoregressive | nothing repairs drift |
| B | Context window (N = 24) | re-grounds perfectly inside the window, not at all outside it. Given a noise-free encoder — strictly stronger than conditioning on N real frames. Still a hard horizon |
| C | Compressed latent memory (32 slots, 8 objects) | capacity is not the constraint; addressing is. You query it with your own drifted belief, so the query degrades exactly as fast as the thing you are correcting |
| D | Cairn | — |
Condition C's failure is the one worth dwelling on, because it is the argument for the whole project. A compressed latent has no row named "object 7"; the only route back to a memory is content-similarity search, and content is precisely what drifted. Cairn replaces that search with an integer.
5 seeds × 6 absence durations × 4 conditions, 8 objects per room, ~3 minutes on a laptop CPU.
Reproduce with python benchmarks/run.py; raw rows in benchmarks/results/.
Fraction of episodes where the object came back drawn, with the same persistent identity, and within 0.5 m of where the run itself last showed it. Higher is better.
| absence (frames) | (A) Vanilla | (B) Context-window | (C) Compressed-memory | (D) Cairn |
|---|---|---|---|---|
| 4 | 0.80 | 1.00 | 0.60 | 1.00 |
| 8 | 1.00 | 1.00 | 0.40 | 1.00 |
| 16 | 0.80 | 1.00 | 0.20 | 1.00 |
| 32 | 0.20 | 1.00 | 0.20 | 1.00 |
| 64 | 0.20 | 0.20 | 0.00 | 1.00 |
| 128 | 0.00 | 0.00 | 0.00 | 1.00 |
The same episodes as a distance. Note B's shape: flat until its 24-frame window runs out, then it becomes A.
| absence (frames) | (A) | (B) | (C) | (D) Cairn |
|---|---|---|---|---|
| 4 | 0.346 | 0.048 | 0.426 | 0.039 |
| 8 | 0.260 | 0.047 | 0.593 | 0.039 |
| 16 | 0.370 | 0.047 | 0.758 (60% drawn) | 0.034 |
| 32 | 0.590 | 0.197 | 0.604 (40% drawn) | 0.038 |
| 64 | 0.815 | 0.747 | 1.480 (60% drawn) | 0.036 |
| 128 | 1.225 (80% drawn) | 0.970 (80% drawn) | 2.289 (40% drawn) | 0.036 |
(m; n% drawn = the object was missing entirely in the rest. A low error over few drawn
objects is not a good result, which is why the success table above is the headline and this
one is the supporting detail.)
| metric | (A) | (B) | (C) | (D) Cairn |
|---|---|---|---|---|
| position error vs. ground truth (m) | 0.549 | 0.311 | 0.898 | 0.075 |
| object drawn on return | 0.97 | 0.97 | 0.67 | 1.00 |
| ID-switch rate | 0.011 | 0.008 | 0.026 | 0.005 |
| object disappearance rate | 0.022 | 0.007 | 0.092 | 0.038 |
| integrated consistency debt | 11.8 | 8.8 | 19.5 | 4.9 |
| debt slope (per frame) | +0.0079 | +0.0012 | +0.0192 | +0.0024 |
| FVD-proxy (lower better) | 0.079 | 0.013 | 0.131 | 0.068 |
| temporal flicker | 0.0230 | 0.0233 | 0.0217 | 0.0235 |
| total ms/frame | 6.7 | 6.5 | 6.0 | 9.5 |
| ledger bookkeeping ms/frame | 0.35 | 0.35 | 0.40 | 2.69 |
The ground-truth row matters most for trusting the rest: Cairn could in principle be self-consistently wrong — a ledger that recorded a fiction and then enforced it. It is not. It is also 4–12× closer to the true world than any baseline.
Debt spikes for everyone when the camera comes back — that is what a return is. The difference is what happens next: Cairn's control loop fires and the debt collapses, while A/B/C stay elevated because they have nothing to correct with. The controller is inert for them by construction, not by oversight.
| edit | (A) | (B) | (C) | (D) Cairn | (D) landed in ledger |
|---|---|---|---|---|---|
move |
n/e | n/e | n/e | 1.00 | 1.00 |
remove |
n/e | n/e | n/e | 1.00 | 1.00 |
set_attr |
n/e | n/e | n/e | 1.00 | 1.00 |
n/e = not expressible. A–C are not bad at edits; the operation is undefined for them,
because there is no row to write to. Reported as a structural gap rather than a measured
score. Compliance is graded from the generated pixels, never from the ledger, and requires
both that the new state is present and that the old state is gone — otherwise a model
could pass by drawing the object twice.
| mechanisms | t=16 | t=64 | t=128 | position err | appearance err |
|---|---|---|---|---|---|
none (ledger built, never read) |
0.80 | 0.20 | 0.00 | 0.773 | 0.112 |
i appearance conditioning |
0.80 | 0.40 | 0.20 | 0.653 | 0.039 |
i+ii + anchor inpainting |
1.00 | 1.00 | 1.00 | 0.035 | 0.029 |
i+ii+iii + verify & resample |
1.00 | 1.00 | 1.00 | 0.036 | 0.020 |
Read honestly:
nonereproduces condition A exactly. The ledger by itself does nothing — it has to be read back and enforced.- (i) cuts appearance error ~3× and barely moves position error. It fixes what the object looks like, not where it is.
- (ii) is the load-bearing mechanism. Position error drops ~20×; the curve goes flat. Everything in the headline figure comes from this row.
- (iii) does not pay off measurably at this difficulty. It fires often and refines
appearance slightly, but (i)+(ii) already saturate the benchmark — and keep saturating it
even when 45% of conditioning is silently discarded (
--only ablationruns that stress). Its value is insurance against samplers with weaker adherence. Presenting it as a contributor here would be easy and misleading.
Cairn adds 2.7 ms/frame of bookkeeping. Against this repository's numpy rasteriser that
is 28% of runtime, which sounds alarming and means nothing: a real video diffusion step costs
O(100 ms)–O(1 s), against which 2.7 ms is a rounding error. The absolute milliseconds are
the claim; the percentage is an artefact of how cheap the surrogate is. The cost is per
chunk, not per denoising step, and commit_stride trades it down further.
Generation quality is unchanged (FVD-proxy 0.068 vs 0.079 for vanilla; flicker within 2%) — the falsification test for "forcing is free". If latent compositing left seams or duplicated objects, both numbers would rise.
The experiment is only worth as much as its ability to fail, so:
- No oracle. Everything the ledger knows arrives through
cairn.commit.detect_objectsrunning on pixels the generator emitted, with the camera pose (a control input a world model always has) as the only side information. A test asserts that relabelling the scene graph cannot change a detection. - The absence interval is measured, not assumed. After building each path we rasterise the true scene and count the frames in which the target is actually off screen.
- The return viewpoint is never the departure viewpoint. The camera translates while away, so no method can pass by replaying its last frame of the object.
- The scene fights re-identification. Two near-duplicate colour pairs and repeated shapes, asserted by a test — a room of eight obviously different objects would make ID switches vacuous.
- The baselines are steel-manned, and their degradation is asserted by tests. If a change
ever made the world easy,
test_baselines_degrade_with_absence_durationfails too. - Episodes that cannot be well posed are excluded, not scored. An object permanently
occluded from the room centre never gets a clean look, so remembering it is not a
meaningful ask;
pick_targetskips those rather than charging a framing accident to memory.
Worth recording because each was a real bug with a general lesson:
- A closed perception→storage→generation loop amplifies estimator bias. Cairn renders stored state, perceives the render, and commits the result — so a 1.7% systematic size bias compounds to 2× over 128 re-observations. Fixed by making the update a recursive mean (gain ~ 1/n) instead of a fixed-gain filter, and by down-weighting observations of regions Cairn itself just anchored: fusing your own prediction as a measurement is the classic error, and a closed loop makes it unstable rather than merely inefficient.
- Conditioning must describe the local world, not just the visible part. Anchoring only
the on-screen rows leaves an object the ledger has moved out of view still sitting in the
model's context at its old place — so the frame shows it twice. See
RetrieveConfig.context_radius. - Forcing permutes the generator's internal slots, which silently corrupts naive
evaluation. When the ledger anchors a row onto whichever slot matches by content, the
sampler may reuse a slot that began life as a different object. Nothing is wrong with the
frame; an evaluator tracking objects by slot id starts scoring the wrong one and reports
catastrophe. Fixed with an explicit, non-circular custody chain (
_custody_track).
- The generator is a surrogate. It is a small explicit model of AR video drift (random
walk + prior pull + salience decay), not LTX-Video. That is a deliberate trade: exact ground
truth and 120 runs in three minutes, in exchange for not being a real backbone. The
heavyweight path is real code
(
cairn/generators/video_backend.py,CairnPipeline.from_pipeline) but is not what produced these numbers. - FVD-proxy is not FVD. It is a Fréchet distance in a fixed random-feature space with temporal-difference channels. It detects compositing artefacts, which is what the quality claim needs; it is not comparable to published FVD.
- The world is 2.5D and rigid. Floor-standing objects, no articulation, no deformation, no lighting change. Everything about which representation to store would need rethinking for a real scene; nothing about the argument for storing it explicitly would.
- The ledger assumes a static world between edits. Adaptivity to genuine unmodelled change is traded away for stability against estimator bias (finding 1).
- Residual edit failures are perception, not obedience. Where compliance is below 1.0, the
edit reached the world state (
landed in ledger= 1.00) but two objects landed on overlapping pixels and the grader could not separate them.
cairn/
ledger.py WorldLedger: rows, transactions, rollback, extent triangulation
commit.py CommitPath: segment → measure → re-identify → merge/register
retrieve.py RetrievePath: the three forcing mechanisms + CairnPolicy
debt.py ConsistencyDebt scalar + threshold controller
edit_api.py move / remove / set_attr / undo, and automatic compliance scoring
metrics.py return consistency, ID switches, disappearance, FVD-proxy
runner.py the experiment driver — where all four conditions are made equal
world/ 2.5D renderer, scene generator, departure→return trajectories
generators/ surrogate generator, baseline memory policies, latent codec
pipelines/ CairnPipeline.from_pipeline(pipe)
cli.py cairn demo | ledger | edit | bench | backends
benchmarks/run.py the five questions, the figures, the report
scripts/build_bench.py the HF dataset (departure→return episodes with exact ground truth)
app.py Gradio app (local): leave & return · edit the world · the ledger
space/ the same demo as a static, browser-only Space (Pyodide, no server)
browser_app.py its compute layer — imports the cairn wheel, returns HTML + PNGs
build_space.py generates index.html from it; `python space/build_space.py --serve`
tests/ 114 tests
Every public function carries a Claim: line naming which assertion it exists to
demonstrate — R return-consistency, B debt-suppression, E editability,
O low-overhead.
Diffusion acceleration (caching, distillation, step reduction) is deliberately untouched; it is a saturated area and orthogonal to this question. Latent memory compression appears only as baseline C — the contribution is the explicit ledger, not a better squeeze.
python scripts/build_bench.py --out data/ # write locally
python scripts/build_bench.py --push NagaYu/cairn-departure-returnDeparture→return episodes with exact per-frame ground truth, measured absence intervals, and scheduled off-screen edits — so the headline result can be reproduced, or contested, without this repository's generator.
@software{cairn2026,
author = {NagaYu},
title = {Cairn: Subordinating Video Generation to an Explicit World Ledger},
year = {2026},
note = {Return consistency that is O(1) in absence duration, and an editable world.},
url = {https://github.com/NagaYu/cairn}
}- Code — https://github.com/NagaYu/cairn
- Demo — https://huggingface.co/spaces/NagaYu/cairn
- Dataset — https://huggingface.co/datasets/NagaYu/cairn-departure-return
MIT.