#602 stage B: rewire upscale-clip and refine-clip through fit/restore

[dkackman]

Stage B of #602. Section Stages → Stage B: rewire upscale-clip and refine-clip of plan v4; the plan comment stays authoritative, including The design, Templates (stage B), Risks and Decisions. Needs stage A (#631, verified).

Filed by the lead agent, model claude-opus-5-5 via anthropic. Reshaped for plan v4 (reconciled 2026-10-06 by the same model): v2's ref_width/ref_height breaking change is gone (Q2′). upscale-clip keeps width/height as the output size, and the stage now also changes fit_to_model itself: a downscale argument, and video as an fps-carrying float32 ndarray. No breaking-change.

What it builds

• In dw/tasks/fit.py:
  • fit_to_model gains downscale: an optional positive integer, default 1. It fits into width/downscale × height/downscale and records that as the model size. It refuses, naming both values, when width or height isn't divisible by it. Its get_task description says so.
  • fit_to_model's video becomes a float32 numpy.ndarray subclass carrying .fps (in dw/media_types.py, alongside AudioVideo), in place of an AudioVideo. Then previous_result:fit.video feeds a pipeline argument or a condition's frames directly.
  • Unit tests for both. One realizes previous_result:fit.video into a pipeline's arguments and a condition's frames through the real argument path, not a mock. Write that one first. If a condition's frames still won't take it, stop and re-plan (plan Risks); don't bend the engine.
• In both upscale-clip and refine-clip, the same step sequence: fit (fit_to_model, mode from a new fit variable, default letterbox) → the pipeline fed previous_result:fit.video → restore (restore_to_source, fit: previous_result:fit.fit) → the existing pair_audio.
• upscale-clip: width/height stay the output size (default 960×544, 32n) and go to the pipeline as today. fit gets width: variable:width, height: variable:height, downscale: 2, so the reference is 480×272, and diffusers' crop becomes a no-op. The IC reference is previous_result:fit.video in place of variable:source_video.
• refine-clip: fit to width×height. loop_frames is replaced, and the upsampler's stretch becomes a no-op.
• If the GPU runs show an edge halo at the letterbox edge, flip the default to stretch (Q1).

What it owes

• Rewritten descriptions and summaries in both templates, within the COMPACT_BUDGET (tests/test_catalog_structure.py:411). They state the changed behaviour: a different aspect is letterboxed and restored, a short source is held and trimmed, and the output is exactly 2× the source at the source's length. They also say which size each template's variables name: the output for upscale-clip, the working size for refine-clip.
• workflows/templates/ltx2/README.md, plugins/dw/skills/ltx-2.5/SKILL.md with a plugin version bump, and docs/RECIPES_24GB.md:186.
• Updated tests that pin today's shape: tests/test_ltx2_ic_loras.py (:212, :174-185 for upscale-clip only, :216), tests/test_catalog_structure.py:566-645 (TestLtxRefineClip), and the tests/test_step_cache.py:706-726 hashes if the configuration changes.

Deploy

Server, plus a plugin release.

Passes when

Its own repro works, per the plan's stage B acceptance intent:

• both templates validate clean on a 640×480, 50-frame, 24 fps asset with a soundtrack;
• final/ is 1280×960, 50 frames, 24 fps, with audio at the source's length, no edge content lost, no bars, and no lapped tail;
• intermediate/ shows the model frame with bars;
• a source already at the working size gives the same output size as before;
• downscale: get_task shows it, width=512, height=288, downscale=2 gives a 256×144 fit with that model size in the record, and downscale=3 with width=512 is refused naming both;
• the existing refusals are unchanged;

and every pending: #632 case the tester writes passes.

After it lands (lead files to Don)

• The Q3 follow-up (adopt the pair in restore-deblur / restore-decompression), once letterbox holds up in these runs.
• Removal of loop_frames as a proposal, if no template consumer is left.

[dkackman]

Stage B stopped before any code: the approved plan (v2) doesn't hold against the code. The approval on #602 is withdrawn, and plan v3 goes to Don for review. No branch was pushed, because nothing was built.

Two things break:

1. Q2's ref_width/ref_height shape can't be wired. LTX2InContextPipeline takes width/height at the output size and derives the reference size itself: ref_width = width // reference_downscale_factor (pipeline_ltx2_ic_lora.py:1403-1404). With only ref_width/ref_height variables, nothing can pass it 2 × ref_width:

   • workflow JSON has no expressions;
   • no task does arithmetic;
   • variable_constraints has no rule relating two variables (SNAP_FIELDS, dw/variable_constraints.py:57).

   Setting reference_downscale_factor: 1 instead would give a 1× output from a LoRA trained for 2×.

2. previous_result:fit.video can't feed a pipeline.

   • It resolves to the AudioVideo that fit_to_model returns (dw/tasks/fit.py:183).
   • Nothing unwraps an AudioVideo for a diffusers argument or for a condition's frames. generative-upscale and window_video consumers read .frames off a bare AudioVideo, but here it sits inside a dict, and get_artifact_properties doesn't walk video.frames (dw/result.py:217).
   • The plan's named fallback (save fit.video as a hidden intermediate and reference it) doesn't work either. output: only names earlier runs, and an h264 round trip would lose detail before a 2× re-render.

Plan v3 proposes a fix for each on #602. This stage waits for Don's decision on it.

Lead agent, model claude-opus-5-5 via anthropic.

[dkackman]

Reshaped for plan v4 (#602). The body now says the stage also changes fit_to_model: a downscale argument, and video as an fps-carrying float32 ndarray. upscale-clip keeps width/height as the output size, with downscale: 2 on fit. v2's ref_width/ref_height is gone, so the breaking-change label is removed. A downscale acceptance item is added. — lead agent, claude-opus-5-5 via anthropic