diff --git a/README.md b/README.md index 62512d7..b7409c9 100644 --- a/README.md +++ b/README.md @@ -26,6 +26,10 @@ composite. MatteLoop is the whole path from that clip to that loop: - **Preview a single frame** to judge the cutout before committing to a render. - **Trim, crop and resize** the finished cut, and watch it loop on screen until it is right. +- **Mark exclusion regions** over burned-in overlays or other unwanted + foreground; those areas are made transparent after segmentation. Turn on + **Blank before the model** when an excluded bystander is distorting the + matte; this re-segments the clip and uses a separate cached cut set. - **Render** a lossless transparent animated WebP. - **Choose or clear the output directory** in the inspector's Output section. - **Replace the open clip** by dropping another video onto the window, or with diff --git a/assets/screenshots/main-window.png b/assets/screenshots/main-window.png index dcbac21..9c2ffd1 100644 Binary files a/assets/screenshots/main-window.png and b/assets/screenshots/main-window.png differ diff --git a/assets/screenshots/model-manager.png b/assets/screenshots/model-manager.png index 59cb9e8..8b7ba95 100644 Binary files a/assets/screenshots/model-manager.png and b/assets/screenshots/model-manager.png differ diff --git a/assets/screenshots/transform-expanded.png b/assets/screenshots/transform-expanded.png index 6c11726..2f83874 100644 Binary files a/assets/screenshots/transform-expanded.png and b/assets/screenshots/transform-expanded.png differ diff --git a/docs/designs/exclusion-regions.md b/docs/designs/exclusion-regions.md new file mode 100644 index 0000000..66f1017 --- /dev/null +++ b/docs/designs/exclusion-regions.md @@ -0,0 +1,1589 @@ +# Design: Exclusion regions + +Architecture pass on 2026-09-12 for issue #80 +Branch: feat/issue-80-exclusion-regions +Repo: smb-org/MatteLoop +Status: Decisions 1–6 are REVISED after an independent adversarial review +and are **implemented on this branch** (`b1caf0a` … `70293e0`). Decision 7 +(pre-model regions) is a second architecture pass on 2026-09-12 and is **not +implemented**; its `file:line` citations were read at `70293e0`, the earlier +decisions' at the commit that preceded their implementation. + +## Review findings addressed + +An independent review rejected the first draft. All nine findings were +re-verified against the code before designing against them; two turned out to +have a smaller root cause than the review prescribed, and one turned out to be +a defect that already ships. + +| # | Finding | Answered by | Verdict on re-check | +|---|---|---|---| +| 1 | The oriented→widget mapping is wrong for anamorphic 0°/180° sources | Decision 2a | **Confirmed, and wider than reported: the crop rectangle is mispainted today.** Root cause is one wrong viewport expression in non-frozen `core/crop.py:260-263`. Scoped out as a prerequisite fix, not folded in. The review's prescription — a new direct mapping for regions — is rejected: it would leave the crop broken and duplicate a transform. | +| 2 | Persisted regions are painted where they do not apply | Decision 1 (*Persistence*), Decision 2b | Confirmed. `build_crop_geometry` slides (`geometry.py:588` → `:387-395`) where `cut_exclusions` intersects. Answered by clipping regions to the source once, on `SourceLoaded`. | +| 3 | One dispatch per pointer move is a worker storm | Decision 6 (*Gesture contract*) | Confirmed, and worse than reported: `_schedule_facts` (`ui/transform_stage.py:412-442`) starts a `QThread` immediately with **no debounce**, and every dispatch rewrites ~12 QSettings keys (`ui/controller.py:486-504`, `ui/preferences.py:44-70`). Answered by commit-on-release. | +| 4 | The empty-union catch swallows corrupt frames | Decision 5 | Confirmed. `INVALID_FRAMING` also covers a non-RGBA frame (`geometry.py:1032-1038`), mismatched canvases (`geometry.py:770-775`) and a plan/frame size disagreement (`geometry.py:802-807`). Answered by a distinct signal, not a caught error code. | +| 5 | The "no re-segmentation" promise is not met | *Constraints* | Confirmed. The promise is narrowed in writing rather than engineered around. | +| 6 | Cache exactness is overstated | Decision 4 (*What this does not achieve*) | Confirmed on all three counts. Only the cheap one is fixed; the other two are documented instead of claimed away. | +| 7 | The downgrade dismissal is too strong | Decision 4 (*Migration*) | Confirmed. Restated as a bounded residual risk with the one guard that is free. | +| 8 | The render-loop copy ignores the ownership contract | Decision 3, site 2 | Confirmed — and dissolved. Staging before masking removes the copy entirely, so there is no second owner to register. | +| 9 | Governance and budget | *Implementation order*, change list | Confirmed. `main_window.py` is 351 lines and the ratchet only applies above 800 (`scripts/check_guardrails.py:124-126`), so it was never a baseline question; that flag is dropped. The scope entry moves first. | + +## Problem statement + +A burned-in overlay (Twitch chat, alert box, webcam frame, a bystander at the +frame edge) is read as foreground and survives into the cutout. `Crop` cannot +remove it when the overlay sits in front of the subject, and a +largest-component filter was measured to fail in both directions (#80). + +The user drags one or more rectangles over the source frame; those areas are +forced to alpha 0 **after segmentation**, as part of `FramingSpec`. Two +decisions are settled by the maintainer and are not revisited here: + +- **Rectangles only, several of them.** No polygon, no ellipse. +- **Post-segmentation by default.** The crop is part of the cut key + (`core/fingerprints.py:212`), so a pre-segmentation mask re-segments the + whole clip on every change, and rembg models judge the frame globally, so + blanking an input region is model-dependent. Zeroing alpha afterwards is + deterministic and reuses the stored cut set. + + **Reopened 2026-09-12, Decision 7.** With the post path alone the model + still reads a bystander at the frame edge as foreground and that distorts + the matte of the wanted subject. A region can now *also* be applied before + the model; the re-segmentation cost is accepted and keyed in the cut. The + post path stays as the default and the instant-feedback path. + +## What already exists (verified) + +**Cut frames are crop-sized.** `_produce_cut_frame` (`jobs/render.py:1529`) +decodes, applies `apply_source_crop` with `request.crop` +(`render.py:1552-1558`), segments, and refuses anything that is not exactly +`(crop.height, crop.width, 4)` (`render.py:1585-1596`). Every stored and every +fresh cut frame is therefore in **cut space**: origin at the crop's top-left. +The crop itself is in oriented source pixels (`core/specs.py:108-109`, +`ui/crop_canvas.py:44`). Claim 1 of the brief holds. + +**Six consumers read cut alpha for framing, not five.** The brief listed five; +the sixth is the one a user looks at most: + +| # | Site | What it does with the alpha | +|---|---|---| +| 1 | `jobs/render.py:924` | preview: `alpha_bounds(cut, …)`; then `apply_framing` at `:929`/`:945` and `_immutable_rgba(cut)` at `:959` — the preview image itself | +| 2 | `jobs/render.py:1109` | render loop: `alpha_bounds` per frame into the range union; the *same* `cut` object is then stored by `self._workspace.stage` at `:1112` | +| 3 | `jobs/render.py:1338` | `_rebuild_union`: `read_cut` → `alpha_bounds` when the cached union misses | +| 4 | `jobs/render.py:1386-1387` | `stage_encoder_frames(partial(read_cut, private), …)` → `_stage_frame` → `apply_framing` (`jobs/transform_stage.py:122-124`) | +| 5 | `ui/transform_stage.py:705-707` | `union_alpha_bounds(_iter_stored_frames(…))` — the Transform group's framed size | +| 6 | **`ui/result_player.py:488-490`** | `_load_one_frame`: `frame_reader.read` → `apply_framing(cut, plan)` — the frames the result player loops | + +Missing site 6 would make the player show the overlay that the encoder +removes; missing site 5 would make the trim readout hug it. + +**All cut pixels enter framing through three producers**, each with a bounded +caller list: `_produce_cut_frame` (sites 1, 2), `WorkspacePort.read_cut` +(`render.py:158`; sites 3, 4; `FilesystemWorkspacePort.read_cut` at +`render.py:538-544` returns `opened.copy()` — `workspace/_models.py:245-247` +— so it is a fresh object per call), and `FrameReader.read` +(`ui/result_player.py:55-58`; sites 5, 6; `_DirectFrameReader` at +`ui/transform_stage.py:60-68` opens the file per call). Nothing else in `src/` +calls `alpha_bounds`, `union_alpha_bounds` or `apply_framing` (grep on this +branch). + +**The union cache.** `CutUnionMetadata(bounds, alpha_threshold, fingerprint)` +(`workspace/_manifest.py:112-135`) is written at `render.py:1126-1134` and by +CAS at `render.py:1348-1363`; the manifest parser refuses any other key +(`_exact_keys` at `_manifest.py:361-365`), so the format cannot grow. +`union_fingerprint` (`core/fingerprints.py:249-260`) hashes only `cut_key` and +`alpha_threshold`. It is checked in **three** places that do **not** agree: + +- `PreviewService._matching_union` (`render.py:989-997`): fingerprint **and** + threshold text. +- `RenderService._rebuild_union` (`render.py:1326-1334`): fingerprint **and** + threshold text. +- `ui/transform_stage.py:_resolve_union` (`:695-703`): **threshold only** — + it never looks at `metadata.fingerprint`. Claim 3 of the brief is therefore + understated: the UI cache would go stale with exclusions *and* is already + blind to any other fingerprint drift. + +External edits already drop the cache (`workspace/_cut_ops.py:380`), and the +UI skips it for edited manifests (`ui/transform_stage.py:695-696`). + +**`FramingSpec`** (`core/specs.py:244-291`) is a frozen dataclass with +`validate()`; it is rebuilt from flat `ParameterState` fields in three places: +`core/parameters.py:72-77` (validation), `ui/request_builder.py:97-102` (the +request), `ui/transform_stage.py:723-729` (the facts worker). Claim 4 holds. + +**Empty mask today.** `union_alpha_bounds` raises `INVALID_FRAMING` +(`core/geometry.py:781-786`), `framing_plan` raises when `trim` and the union +is `None` (`jobs/transform_stage.py:40-45`), and the render aborts — +`tests/jobs/test_render.py:1525-1552` pins that. The preview does not abort: +`local_bounds` becomes `None` and the display is `cut.copy()` +(`render.py:924, 941`). The UI facts worker fails and disables the Transform +group (`ui/transform_stage.py:123, 486-490`). + +**The crop editor.** `CropCanvas` (`ui/crop_canvas.py`, 359 lines) edits one +rectangle: `build_crop_geometry` (`core/geometry.py:570-615`) fans one +`CropSpec` out into eight handles plus a `"crop"` body with a fixed priority +(`:601-611`); `crop_from_drag`/`nudge_crop` (`core/crop.py:33-71`) move or +resize any rectangle given a target name; the two hooks `_constrain` and +`_crop_event` (`crop_canvas.py:261-280`) let `ResultPlayerCanvas` reuse the +whole machinery for a different rectangle and a different command +(`result_player.py:253-265`). `set_crop_edit` (`result_player.py:166-190`) is +the precedent for a mode switch on this canvas. + +**Frozen modules on this path** (`scripts/guardrails-baseline.json`): +`jobs/render.py` 2886, `core/geometry.py` 1141, `core/state.py` 882, +`ui/inspector.py` 897. Growth for a real feature needs the maintainer's +agreement and `check_guardrails.py --update` +(`docs/engineering-guardrails.md:279-303`). + +The ratchet only fires **above** the 800-line module budget +(`scripts/check_guardrails.py:124-126`), so a module under 800 is not frozen +however long its baseline entry is. `ui/main_window.py` is 351 lines: the +first draft listed it as frozen at 328 and that was wrong on both counts. + +## Constraints + +- Post regions apply after segmentation and never enter the cut key. A + post-region change must not invalidate the stored cut set, and must + invalidate the preview, the render fingerprint and the union cache. + **Pre-model regions (Decision 7) are the one exception by design:** they + enter the cut key, and only they do; a user without them keeps the key + 0.4.1 wrote. +- **The re-segmentation promise, stated precisely** (review finding 5). What + holds: the stored cut set stays valid under any regions, so **Render and + Rebuild never re-segment** for a region change — they reread the stored + frames and re-measure. What does **not** hold: the *preview* is invalidated + like any other cleanup change, and `AppState` keeps only the displayed + `QImage` (`core/state.py:72`, `ui/presenter.py:177`), not the cut behind it, + so the next Preview press reruns the model on that one frame. Retaining the + unmasked preview cut in order to re-frame it locally is a larger change than + this feature (it is a preview-caching feature in its own right) and is + recorded in *Rejected alternatives*. Nothing in this document may claim that + a region change is free for the preview. +- No new dependency; Pillow already does everything needed. +- `core/geometry.py` is not touched. Everything new lives in one new + Qt-free module and the non-frozen files. +- Degrade, never refuse (G4, `docs/engineering-guardrails.md:189`). +- Preferences, cut sets and union metadata written by v0.4.1 keep working. +- String literals at every `translate()` call site; German catalogue entries; + README and generated screenshots updated (`CLAUDE.md`, *Keeping the README + honest*). + +## Decision 1 — Data model + +`FramingSpec` gains one field: + +```python +@dataclass(frozen=True) +class FramingSpec: + trim: bool = False + alpha_threshold: Decimal = Decimal("2.0") + padding: int = 0 + stretch_x: Decimal = Decimal("1.0") + exclusions: tuple[CropSpec, ...] = () +``` + +- **Type: `tuple[CropSpec, ...]`.** A region is an axis-aligned integer + rectangle in oriented source pixels — exactly what `CropSpec` already is + (`specs.py:108-134`). `crop_from_drag`, `nudge_crop` and `clamp_crop` + (`core/crop.py:33-90`) are pure rectangle arithmetic and take a region + unchanged. No `ExclusionRegion` type. + + **Correction (review finding 1).** The first draft also claimed + `build_crop_geometry` works on a region "with no adaptation". That is + wrong, and not only for regions: the oriented→raw helper it is fed from is + broken today. See Decision 2a — it is a prerequisite fix, not part of this + feature. +- **Validation** (`FramingSpec.validate`): `exclusions` must be a `tuple`, + every element a `CropSpec` (which already enforces `x, y >= 0`, + `width, height >= 1`). No upper bound against the source: a region beyond + the frame is clipped at apply time (Decision 2), and `validate_for_source` + (`specs.py:665-674`) is left alone so a region persisted from a larger + video does not refuse a smaller one. No count cap. +- **Ordering: none on the spec.** The tuple keeps user order, because the + canvas selects a region by index and a reorder under the pointer would be a + visible jump. Canonicalisation happens once, in `cut_exclusions()` + (Decision 2): sorted, de-duplicated, clipped. Zeroing is commutative, so + two orderings of the same regions are the same request everywhere it + matters — the fingerprint, the union cache, the pixels. +- **Fingerprint encoding.** The *effective* cut-space boxes, never the raw + regions: `"exclusions": [[left, top, right, bottom], …]` sorted ascending, + **present only when non-empty**. Rationale in Decision 4. +- **Reducer.** `ParameterState.exclusions: tuple[CropSpec, ...] = ()` + (`parameters.py:47-60`), validated by passing it into the existing + `FramingSpec(...)` call at `:72-77`. One new event, + `ExclusionsChanged(exclusions: tuple[CropSpec, ...])`, reduced like + `_reduce_cleanup_value` (`:322-337`): reject anything that is not a tuple of + `CropSpec`, **no-op when the effective boxes are unchanged** (not merely + when the raw tuples are equal — Decision 4), otherwise + `PreviewInvalidationReason.CROP_CLEANUP`. `ParametersReset` clears it + (`:218-252`, add to the reset and to the CROP_CLEANUP comparison). +- **Persistence.** A parameter, like the alpha threshold: it survives a + source change and a restart. The overlay of a streamer's layout is the same + on every clip of that stream, which is the case worth carrying. One + QSettings string `parameters/exclusions` = `"x,y,w,h;x,y,w,h"`; + `parameters_from_values` (`parameters.py:393-420`) parses it with the + per-key fallback the file already uses: any malformed entry → `()`. + + **Correction (review finding 2).** The first draft called a stale rectangle + on an unrelated video "harmless". It is not: a carried region that + *intersects* the new frame deletes those pixels, and one that sticks out is + **painted somewhere else than it masks**, because `build_crop_geometry` + slides a rectangle inward (`geometry.py:588` → `clamp_source_rect`, + `geometry.py:387-395`) while `cut_exclusions` intersects. Measured: a + carried `(1700,0,100,100)` on a 640-wide source paints at x 540…640 and + masks nothing. + + Carrying stays, but the regions are **clipped to the new source once, on + load** (Decision 2b), so a carried region is either a real rectangle inside + the frame or gone. Painted and masked can then never disagree, because + after the clip `clamp_source_rect` has nothing left to slide. + +## Decision 2 — Coordinates: stored in oriented source space, mapped once + +Regions are stored in **oriented source pixels**, the crop's own space. A +burned-in overlay is fixed in the source frame; if regions lived in cut space, +moving the crop would drag them along. The brief's claim 1 is right. + +The one mapping lives in a new Qt-free module, `src/matteloop/core/exclusion.py`: + +```python +def cut_exclusions( + exclusions: tuple[CropSpec, ...], crop: CropSpec +) -> tuple[PixelBounds, ...]: + """Map oriented-source regions into the crop-sized cut, clipped and canonical.""" + boxes: set[tuple[int, int, int, int]] = set() + for region in exclusions: + left = max(region.x, crop.x) - crop.x + top = max(region.y, crop.y) - crop.y + right = min(region.x + region.width, crop.x + crop.width) - crop.x + bottom = min(region.y + region.height, crop.y + crop.height) - crop.y + if right > left and bottom > top: + boxes.add((left, top, right, bottom)) + return tuple(PixelBounds(*box) for box in sorted(boxes)) + + +def exclude_alpha(image: Image.Image, boxes: tuple[PixelBounds, ...]) -> None: + """Zero RGBA inside each box, in place.""" + for box in boxes: + image.paste((0, 0, 0, 0), (box.left, box.top, box.right, box.bottom)) +``` + +- A region **fully outside the crop** produces no box: it is absent from the + pixels, from the union cache key and from the render/preview fingerprints. + Dragging it stales nothing — but only because the reducer compares effective + boxes rather than raw tuples (Decision 4, false miss 1). The first draft + claimed this while the reducer still compared raw tuples, and was wrong. +- A region **partly outside** is clipped to its intersection. +- `PixelBounds` (`geometry.py:142`) already refuses empty or negative boxes, + so the `right > left and bottom > top` guard is the only check needed. +- RGB is zeroed with the alpha, matching the canonical black that + `_decontaminate_edge_colors_in_place` writes under alpha 0 + (`render.py:1611-1620`), so the two edge modes produce identical bytes in an + excluded box. +- Alpha 0 is never "visible": the threshold table is strict + (`geometry.py:1108-1112`), so an excluded box drops out of every bound at + every threshold including 0. + +### Decision 2a — Prerequisite: the oriented→raw mapping is broken today + +**This is a defect that already ships, not one this feature introduces.** + +`oriented_rect_to_source_rect` (`core/crop.py:94-108`) builds an intermediate +`MediaTransform` through `_orientation_transform` (`core/crop.py:254-266`), +whose viewport is: + +```python +viewport=SizeF( + source_height if rotation in {90, 270} else source_width, + source_width * pixel_aspect if rotation in {90, 270} else source_height, +), +``` + +That expression is `MediaTransform.oriented_display_size` +(`core/geometry.py:305-316`) written out by hand — correct for 90°/270°, and +**missing the `* pixel_aspect` for 0°/180°**. The intermediate transform then +fits a 32-wide display into a 16-wide viewport, scale 0.5 instead of 1, and +stops round-tripping. + +Measured on this branch over rotation {0, 90, 180, 270} × PAR {1, 2, 1.5, +0.75}, comparing the painted `visual["crop"]` against the oriented rectangle's +true position in `content_rect`: **6 of 16 combinations are wrong — every +0°/180° case with PAR ≠ 1.** The review's worked example reproduces exactly: +coded 16×8, PAR 2, rotation 0, oriented rect (8,0,8,8) → raw +`RectF(8,-4,8,16)`. + +The single existing test pins rotation 270 with PAR 2 +(`tests/core/test_crop.py:63-70`) — one of the ten passing combinations, which +is why this was never caught. + +**The consequence today**, before any region exists: `crop_canvas.py:288` +feeds this helper, so **on an anamorphic un-rotated source the crop rectangle +is painted in the wrong place and at the wrong size.** Anamorphic 0° is not +exotic — it is DVD-era and broadcast 16:9 SD content. + +**The fix is one expression** in the non-frozen `core/crop.py`: make the +viewport the oriented display size for every rotation. Verified on this +branch — all 16 combinations pass. + +**Scoping.** It is a pre-existing defect with its own trigger, so it lands +**before** this feature, on its own branch, as its own `fix:` commit with its +own `Trigger:` line and its own issue. Exclusion regions depend on it and must +not be implemented on top of the broken helper — but they must also not +smuggle the fix in as a detail of a feature commit. + +Once it is in, regions genuinely do reuse `build_crop_geometry` unchanged, +which is what the first draft assumed without checking. + +### Decision 2b — Carried regions are clipped to the source on load + +`SourceLoaded` (`core/state.py:398-407`) already resets the crop to the full +frame via `default_crop_for_source`. Regions are carried instead of reset, so +they are clipped there in the same `replace(...)` call: + +```python +parameters=_parameters.clip_exclusions(state.parameters, event.value), +``` + +`clip_exclusions` lives in the non-frozen `core/parameters.py` and intersects +every region with `(0, 0, width, height)`, dropping what falls entirely +outside — the same intersect-don't-slide rule as `cut_exclusions`. + +- **Why intersect, not `clamp_crop`.** `clamp_crop` *slides* a rectangle + inward to preserve its size (`core/crop.py:73-90`, deliberately, for issue + #25). Sliding an exclusion moves it onto content the user wants to keep — + the opposite of what it is for. Regions intersect; the crop keeps sliding. +- **Why it cannot be bypassed.** Every source that reaches a READY state goes + through `SourceLoaded`; there is no other writer of `source_value`. A region + that survives into `state.parameters` is therefore inside the frame by + construction, which is what makes painted == masked a property rather than a + convention. +- **Cost.** `core/state.py` is 882 lines and ratcheted, so this is **+1 line + on a frozen module** and needs the maintainer's agreement plus + `check_guardrails.py --update`. That is the honest price of the invariant; + the alternative (clipping only for display) is in *Rejected alternatives* + with the drag-jump it causes. +- Clipping on load, not on every read, also means the fingerprint sees the + same rectangles the user sees. + +**Which crop.** Preview, render and rebuild use `request.crop` — a rebuild +finds its cut set by a key that includes that crop +(`render.py:561-591`, `fingerprints.py:207`), so the two are equal by +construction. The UI transform stage has no request; it reads the cut's own +crop from `manifest.cache_key_inputs["crop"]` with a `_crop_from_manifest` +helper beside the existing `_fps_from_manifest` +(`ui/transform_stage.py:741-748`), which reads the same mapping. + +## Decision 3 — The seam: one function, three producers, six sites + +`exclude_alpha` is the only function that zeroes, and `cut_exclusions` the only +function that maps. There is no *single call site* — the pixels arrive through +three producers, and the render loop must measure an excluded frame while +storing the unexcluded one — so the argument is a bounded enumeration, not a +choke point: + +| Site | Change | +|---|---| +| 1 preview (`render.py:913-960`) | `exclude_alpha(cut, boxes)` once, right after `_produce_cut_frame`. Bounds, both `apply_framing` calls and both `_immutable_rgba` images see it. | +| 2 render loop (`render.py:1108-1112`) | **stage first, then mask in place** — see below. No copy. | +| 3 `_rebuild_union` (`render.py:1338`) | `exclude_alpha(image, boxes)` after `read_cut`. | +| 4 encoder (`render.py:1386`) | `stage_encoder_frames` gains `exclusions: tuple[PixelBounds, ...]` and `_stage_frame` calls `exclude_alpha(cut, exclusions)` before `apply_framing` (`jobs/transform_stage.py:122-124`). | +| 5 UI facts (`ui/transform_stage.py:710-720`) | `_iter_stored_frames` takes the boxes and calls `exclude_alpha` before `yield`. | +| 6 result player (`ui/result_player.py:480-490`) | `FrameLoadWorker` and `_load_one_frame` take the boxes; `exclude_alpha` before `apply_framing`. | + +**Site 2 in detail (review finding 8).** The first draft masked a `cut.copy()` +so the unmasked frame could still be staged. The review was right that a +second full-resolution RGBA owner must be registered with +`RgbaOwnershipTracker` (`core/rgba.py:72-95`), whose peak and current counts +are reported in `RenderArtifact`. + +There is no copy to register, because the order can simply be swapped. Today: + +```python +bounds = alpha_bounds(cut, request.framing.alpha_threshold) # :1109 +union = _union_bounds(union, bounds) # :1110 +frame_records.append(self._workspace.stage(staged, index, cut)) # :1112 +``` + +Becomes: stage, then mask `cut` in place, then measure. `stage_cut` +(`workspace/_cut_ops.py:56-100`) opens the frame file and writes the PNG +synchronously inside the call and returns a `CutFrame` record derived from the +written bytes — it retains no reference to the image — so mutating `cut` +afterwards cannot reach the stored file. The existing `finally: cut.close()` +(`render.py:1113-1116`) is unchanged. + +Result: one owner, unchanged peak accounting, no tracker change, and one fewer +full-resolution allocation per frame than the first draft proposed. The stored +set still contains unmasked pixels, which is the property the whole design +rests on. + +Why the seam is not lower: `read_cut`/`FrameReader.read` also serve identity +work — hashing and rescanning in `workspace/_cut_ops.py`, external-edit +detection, the CAS's `_scan_cut_set` — where the bytes must be the stored +bytes. Zeroing inside the reader would make an edited-cut scan disagree with +the file on disk. Why not higher (inside `FramingPlan`/`apply_framing`): +bounds are computed before any plan exists, so the four measure sites would +still be explicit, and it grows the frozen `geometry.py`. + +What makes "missed a site" impossible in practice: the docstring of +`core/exclusion.py` lists the three producers and six sites, and the test plan +has one behavioural test per site that fails when that site is skipped. A new +consumer of cut alpha is a new caller of one of three functions, which a grep +shows. + +## Decision 4 — Union-cache invalidation and migration + +The manifest format cannot gain a key (`_manifest.py:361-365`), and it does not +need one: the invalidation goes through the `fingerprint` field that already +exists. + +`core/fingerprints.py`: + +```python +def cut_union_fingerprint( + framing: FramingSpec, crop: CropSpec, *, cut_key: str +) -> str: + payload: dict[str, object] = { + **_schema("cut-union"), + "cut_key": _validated_sha256(cut_key, "cut_key"), + "alpha_threshold": _canonical_decimal(framing.alpha_threshold), + } + boxes = cut_exclusions(framing.exclusions, crop) + if boxes: + payload["exclusions"] = [[b.left, b.top, b.right, b.bottom] for b in boxes] + return _canonical_hash(payload) + + +def union_fingerprint(request: RenderRequest, *, cut_key: str) -> str: + request = _validated_request(request) + return cut_union_fingerprint(request.framing, request.crop, cut_key=cut_key) +``` + +`_framing()` (`fingerprints.py:348-354`) takes the request instead of the spec +and adds the same `"exclusions"` entry under the same non-empty rule, so +`preview_fingerprint` and `render_fingerprint` track effective regions. + +**One predicate for all three checks.** `jobs/transform_stage.py` (already the +module the UI imports `framing_plan` from) gains: + +```python +def cached_union( + metadata: CutUnionMetadata | None, framing: FramingSpec, crop: CropSpec, + cut_key: str, +) -> PixelBounds | None: + if metadata is None or metadata.fingerprint != cut_union_fingerprint( + framing, crop, cut_key=cut_key + ): + return None + return PixelBounds(*metadata.bounds) +``` + +`_matching_union`, `_rebuild_union` and the UI's `_resolve_union` all call it. +The separate `alpha_threshold` text comparison in the two render sites is +subsumed (the fingerprint already hashes the canonical threshold, and both +are written from the same value at `render.py:1130-1132`), and the UI stops +being blind to the fingerprint. Three predicates that had already diverged +become one. + +**Migration — stated explicitly.** Metadata written by v0.4.1 hashes +`{schema, cut_key, alpha_threshold}`. Because the `"exclusions"` key is +**omitted when empty**, a request without regions computes the byte-identical +fingerprint: **existing union metadata stays a hit**, and existing preview and +render fingerprints are unchanged. A request *with* regions computes a +different fingerprint → miss → the existing recompute-and-CAS path +(`render.py:1336-1363`) overwrites the entry, exactly as a threshold change +does today. Removing the regions again is a miss again and a second CAS; +that is one pass over the stored frames, the same cost as any threshold +change, and it needs no second cache slot. + +Preferences: the new key is absent on first launch → `()`; nothing else in +`parameters/` changes. Cut sets: untouched, by design. + +**The downgrade risk, stated properly (review finding 7).** The first draft +dismissed it because "the updater refuses downgrades". That is not a +sufficient argument: it does not stop an older binary that is still installed, +restored from a backup, or run as a portable build against the same cache +directory. + +The concrete disagreement: a new build writes region-derived bounds into the +shared manifest. An old **Render/Rebuild** compares the fingerprint +(`render.py:989-997`, `:1326-1334`), fails to reproduce it, and recomputes the +unexcluded union — correct. An old **UI facts worker** compares the threshold +only (`ui/transform_stage.py:695-703`) and accepts the narrower +region-derived bounds, then applies them to unmasked stored frames. The +Transform group would show a framed size tighter than what that build would +actually encode. + +- **Blast radius:** a wrong readout in one panel of an old build. The bounds + are narrower, never wider, so nothing is encoded that the user cannot see; + and the old build's own render recomputes correctly, so no file is wrong. +- **The guard that is free:** the threshold text is written from the same + value in both builds, so it cannot distinguish them — but this design + already replaces that blind check with `cached_union` in the *new* build + (finding: the UI is blind to the fingerprint on `main` today, independent of + regions). Shipping that fix is what stops the next version of this problem; + it cannot retroactively fix a binary already on disk. +- **Residual risk accepted:** an older build, run by hand against a cache a + newer build wrote regions into, shows a stale framed size in the Transform + group until that cut set is rescanned. Not guarded further; guarding it + would need a manifest key the parser refuses (`_manifest.py:361-365`). + +### What this does not achieve (review finding 6) + +No **false hit** was found on re-check: the crop is in the cut key +(`fingerprints.py:207`), so a crop change already changes `cut_key`, and the +effective-box payload matches the pixels exactly. Ordering and exact +duplicates canonicalise away. Rotation is a source property and changes the +source fingerprint. + +Three **false misses** remain, and are documented rather than claimed away: + +1. **A fully-outside region still stales the preview.** The reducer compares + raw tuples, so dragging a region that produces no box dispatches a real + change. Decision 1's earlier claim that "dragging it does not stale + anything" was wrong about the reducer, though right about the fingerprint. + **Fixed, because it is cheap:** the reducer no-ops when the *effective* + boxes are unchanged, comparing `cut_exclusions(new, crop)` against + `cut_exclusions(old, crop)`. After Decision 2b a fully-outside region only + exists relative to the crop, not the source. +2. **Equivalent geometries hash differently.** One rectangle plus a rectangle + contained in it, or two tiles that exactly cover one rectangle, produce + identical pixels and different fingerprints. Merging overlapping rectangles + into a canonical cover is real geometry code for a cache miss nobody has + measured. **Not fixed.** Cost is one recompute. +3. **An empty union cannot be cached at all.** `CutUnionMetadata` refuses + non-positive bounds (`workspace/_manifest.py:126-129`) and `_rebuild_union` + publishes only a non-empty union (`render.py:1347`), so a legitimately + fully-transparent result is rescanned on every facts run. **Not fixed** — + it needs a manifest key the parser refuses. It is also the pathological + case (the user excluded everything), and Decision 5 makes it visible rather + than fatal. + +## Decision 5 — Empty mask: trim degrades to the full canvas + +A region can cover the whole subject, or the whole frame. Today +`framing_plan` raises and the job aborts (`jobs/transform_stage.py:40-45`, +pinned by `tests/jobs/test_render.py:1525-1552`). The user can fix this from +the UI, so G4 does not strictly apply — but a render that aborts because trim +found nothing is a refusal of the one thing the user asked for, and the +degraded result is trivially correct: + +- `framing_plan(source_size, None, framing)` with `trim=True` returns the + **untrimmed** plan (`global_bounds=None`) instead of raising. The output is + the crop canvas plus padding; if the mask is genuinely empty, every frame is + fully transparent — which is what excluding everything means. +- `RenderService` appends `"trim skipped: no pixel above the alpha threshold + survives the exclusion regions"` to `notes` (`render.py:1033`), the channel + the disk advisory already uses. Nothing in `ui/` displays `notes` today + (grep), so this reaches the log, not a dialog. Not adding a UI channel now. +- **What the user sees:** the preview's Result canvas shows the transparent + frame; the completion summary reports the crop's dimensions rather than a + trimmed size; the Transform group's framed size equals the crop. The + rectangles on the Original canvas are the explanation. +- `tests/jobs/test_transform_stage.py:34` and `tests/jobs/test_render.py:1525` + are updated to pin the new behaviour (test plan). **`test_render.py:1522`'s + `visible_size=10` minimum-dimension branch must survive** — it pins a + different refusal (a trimmed box below `MIN_FINAL_DIMENSION`), which is not + degraded here and must keep failing loudly. + +**An empty mask is signalled, not caught (review finding 4).** + +The first draft had the UI catch `ValidationError` with `INVALID_FRAMING` and +treat it as an empty union. That code is not specific to an empty mask — +`union_alpha_bounds` and its helpers raise the same code for: + +- a non-iterable argument (`geometry.py:748-754`), +- a frame that is not a real Pillow RGBA image (`geometry.py:1032-1038`), +- frames whose canvases disagree (`geometry.py:770-775`), +- and later, in `apply_framing`, a frame that does not match the plan + (`geometry.py:802-807`). + +Failure scenario the blanket catch produces: two externally edited frames have +different dimensions. The facts worker swallows the mismatch as "empty", +reports the full manifest canvas as a valid framed size and starts the player; +`apply_framing` then rejects the real frame size, and `FrameLoadWorker` catches +only filesystem and Pillow decode errors, so the failure surfaces somewhere +unrelated — or not at all. A corrupt cut set would present as a working one. + +Instead, the emptiness is carried as a value: + +- `core/exclusion.py` gains + `union_alpha_bounds_or_none(frames, threshold) -> PixelBounds | None`, + which pre-checks the one condition that means "empty" and lets every other + `ValidationError` propagate untouched. It lives beside the other new + helpers rather than in the frozen `core/geometry.py`; `union_alpha_bounds` + is not modified. +- `_resolve_union` (`ui/transform_stage.py:691-707`) and `_rebuild_union` + (`render.py:1338`) call it and branch on `None`. Nothing catches + `INVALID_FRAMING` anywhere in this design. +- `framing_plan` then receives `None` with `trim=True` and returns the + untrimmed plan. + +So an empty mask degrades and a corrupt or inconsistent cut set still fails +loudly — the test plan pins both, in the same pair of tests. + +The preview path needs no change: it already shows the cut unframed when no +union matches (`render.py:940-941`). + +## Decision 6 — UI: a mode switch on `CropCanvas`, geometry reused per region + +**Mode switch, not an extended priority list.** `build_crop_geometry` is in the +frozen `geometry.py` and its priority tuple names one rectangle. Extending it +to N regions means N×9 targets, a priority policy for overlaps, and growth in +a frozen file. A mode switch costs nothing in geometry and answers the overlap +question by construction: **in region mode the crop has no hit targets; in +crop mode regions have no hit targets. A region over a crop handle is never +ambiguous because the two are never live at once.** + +**Controls** (Inspector, *Crop & cleanup* section, one form row after +*Horizontal stretch*, label `"Exclusions"`), hosted the way `TransformGroup` +is hosted — a small widget module so the frozen `inspector.py` grows by the +five lines that construct, place, apply and tab-order it: + +`ui/exclusion_controls.py` — `ExclusionControls(QWidget)`: +- `edit_button`: checkable `QToolButton`, text `"Edit regions"`, object name + `exclusion_edit`. Checked = the Original canvas is in region mode. UI-local + state, like `TransformGroup.crop_edit_toggled`; it is not reducer state. +- `clear_button`: `"Clear"`, object name `exclusion_clear`, enabled only when + regions exist; dispatches `ExclusionsChanged(())`. +- `count_label`: `translate("Inspector", "%n region(s)", "", n)` — Qt's + numerus form, literal at the call site. +- Signals: `edit_toggled(bool)`, `command_requested(object)`. +- `apply(presentation, editable)` mirrors `apply_parameters` + (`inspector.py:487-516`): disabled until a source is loaded; leaving + region mode is forced when the controls become non-editable (a running job). +- Wiring: `main_window.py:224-225` already connects `command_requested`; + one more line connects `edit_toggled` to `original_canvas.set_exclusion_edit`. + +**Source-canvas context menu.** `ExclusionCanvas.contextMenuEvent` is the +first context-menu pattern in the application and remains deliberately small. +It builds its actions from the oriented source point under the pointer: + +- Empty canvas: `Exclude this area`, checkable `Edit exclusion regions`, and + `Remove all regions` when at least one region exists. +- Existing region: `Remove this region`, `Remove all regions`, and checkable + `Edit exclusion regions`. + +The menu is unavailable while `capabilities(state).can_edit` is false, and +each mutating action emits exactly one `ExclusionsChanged` command for the +reducer. `Exclude this area` creates a default-sized rectangle derived from +the source dimensions (20% of each dimension), centred on the click and +clamped inside the frame. It does not start a rubber band: beginning a drag +from a menu is unreliable, while the existing handles let the user resize the +newly selected region immediately. The menu handles keyboard-origin context +events at the selected region, or the frame centre when no region is selected. + +The Inspector row remains the discovery surface and the keyboard path. Its +toggle and the menu's checkable toggle stay synchronized in both directions; +the menu is an accelerator, not a replacement for the Inspector controls. +The checkable Edit action changes UI-local mode and emits no +`ExclusionsChanged`; only actions that change the region tuple dispatch that +reducer command. + +**`CropCanvas` in region mode** (`set_exclusion_edit(enabled: bool)`): +- `CropPresentation` gains `exclusions: tuple[CropSpec, ...] = ()` + (`ui/crop_presentation.py:12-21`), filled by `present_crop` from + `state.parameters.exclusions`. The default keeps every existing constructor + call and test valid. +- **Painting, both modes:** every region is painted whenever it exists. The + effective part is derived through `cut_exclusions` and keeps the current + translucent fill (`#E5484D` at alpha 90) with a 1 px solid border. Any + ineffective remainder is faded further and uses a dashed border, so the + distinction is not colour-only; a fully ineffective region is faded in full. + In crop mode the crop keeps its handles. In region mode the crop is drawn as + its dimmed outline **without handles**, and the selected region gets the + eight handles and the focus ring. Handles always follow the full stored + region, not its effective part. +- **Geometry:** `_rebuild_geometry` builds `build_crop_geometry` for + `_edited_rect()` — the selected region in region mode, else the crop. That + is the entire reuse: the same eight handles, the same + `crop_from_drag`/`nudge_crop`, the same accessible announcement, aimed at a + different `CropSpec`. +- **Press** (left button, editable, region mode): + 1. hit-test the selected region's geometry → a handle or `"crop"` starts a + resize/move drag of that region, exactly as the crop's drag today + (`crop_canvas.py:142-167`); + 2. else, if the oriented point is inside another region → it becomes + selected and a move drag starts in the same press; + 3. else → a rubber band starts from the press point. +- **Move:** nothing is dispatched. The region under the pointer and the rubber + band are both painted from canvas-local state. See *Gesture contract*. +- **Release:** exactly one dispatch. A rubber band smaller than 8×8 oriented + source pixels is discarded and dispatches nothing; otherwise + `ExclusionsChanged(regions + (new,))` and the new region is selected. A + move or resize dispatches `ExclusionsChanged` with that index replaced. + Regions are intersected with the source, never slid (Decision 2b). + +**Gesture contract: one dispatch per gesture (review finding 3).** + +The crop dispatches `CropChanged` on every move, and that is affordable +because the crop only re-renders a canvas. A region change is a *framing* +change, and framing changes are expensive on this path: + +- `_schedule_facts` (`ui/transform_stage.py:412-442`) starts a `QThread` and a + `_CutFactsWorker` **immediately — there is no debounce on this path.** The + module's `_debounce_timer` belongs to the frame worker, not the facts + worker. +- A superseded worker is cancelled but deliberately **retained until its + thread finishes**, because dropping the reference used to crash + (`tests/ui/test_transform_stage.py:506`). A drag would therefore accumulate + concurrent retiring threads, each scanning stored frames. +- Every parameter dispatch rewrites the whole preference block — about a dozen + `QSettings.setValue` calls (`ui/controller.py:486-504`, + `ui/preferences.py:44-70`). + +So a 200-sample drag would mean 200 threads, 200 cut scans and ~2 400 settings +writes. + +The precedent is already in the same module: the result-crop editor suppresses +reloads while editing and performs exactly one on exit +(`ui/transform_stage.py:300-315`, pinned by +`tests/ui/test_transform_stage.py:811`). Regions adopt the same shape: + +- `CropCanvas` holds the in-progress rectangle in local state and repaints + itself directly; the reducer sees nothing. +- `mouseReleaseEvent` dispatches once — or not at all, if the gesture was a + discarded rubber band or left the rectangle unchanged. +- Keyboard nudges dispatch per key press. An arrow key is one discrete edit, + not a stream, so no coalescing is needed there; `Escape` during a drag + abandons it without dispatching. + +This is pinned by a test that counts dispatches across a synthetic +press-move×N-release and asserts exactly one. +- **Keys:** arrows nudge the selected region (Shift = 10 px) through the + existing `keyPressEvent` path; `Delete`/`Backspace` dispatch + `ExclusionsChanged` without the selected region; `Escape` clears the + selection. +- **Hooks:** `_crop_event(rect)` returns `ExclusionsChanged(with index + replaced)` in region mode and `CropChanged(rect)` otherwise; `_constrain` + stays identity. In region mode the canvas calls `_crop_event` only on + release, per the gesture contract. `ResultPlayerCanvas` never enters region + mode, so it is inert there. +- **Leaving the mode** clears the selection, restores the crop handles and + keeps painting the regions. +- **Accessibility:** in region mode the description is + `translate("CropCanvas", "Exclusion region %1 of %2: x %3, y %4, width %5, height %6 source pixels")`. +- Numeric fields for a region: not in this landing (see *Not in scope*). + +**New user-visible strings** (all literals, all with German entries): +`"Exclusions"` (form label, via `inspector_label` in `ui/copy.py:632-654`), +`"Edit regions"`, `"Edit exclusion regions"` (accessible name), `"Clear"` +(exists), `"Clear exclusion regions"` (accessible name), `"%n region(s)"`, +the canvas description above, `"Exclude this area"`, `"Remove this region"`, +and `"Remove all regions"` (context-menu actions). + +**README and screenshots.** `scripts/screenshots.py:_state` (`:120`) gets one +region in its `ParameterState` so `main-window.png` shows a rectangle on the +Original canvas; README gains a short paragraph in the crop/cleanup +description; regeneration per `README.md:150-159`. + +## Decision 7 — Pre-model regions: blank the model input, keyed in the cut + +Second architecture pass, 2026-09-12, on top of the landed post path +(`70293e0`). Settled by the maintainer and not re-argued here: a region can +optionally be applied **before** the model sees the frame; that changes the +segmentation input, so it enters the **cut cache key** and a change to it +re-segments the clip; the post path **stays** and keeps its instant feedback. + +The reason it is urgent, in one sentence: with the post path alone the model +still reads the bystander as foreground, and a person at the frame edge +changes how the model resolves the wanted subject — the post mask removes the +bystander's pixels, not the model's opinion of them. + +### 7.1 Data model — one global switch, the rectangles reused + +**Chosen: a global switch.** One new boolean on the parameters and one new +tuple on the segmentation spec; no new rectangle type, no second tuple in the +reducer, no per-region flag. + +```python +# core/specs.py +@dataclass(frozen=True) +class SegmentationSpec: + model_id: str = "birefnet-portrait" + edge_mode: EdgeMode = EdgeMode.STANDARD + alpha_matting: AlphaMattingSpec = field(default_factory=AlphaMattingSpec) + execution_provider: str = CPU_EXECUTION_PROVIDER + exclusions: tuple[CropSpec, ...] = () # regions the model must not see + +# core/parameters.py +@dataclass(frozen=True, slots=True) +class ParameterState: + ... + exclusions: tuple[CropSpec, ...] = () + exclusions_before_model: bool = False +``` + +- **Where each kind lives.** A pre-model region is segmentation input, so the + request carries it on `SegmentationSpec` (`core/specs.py:205-240`), the spec + the cut key already hashes through `model` and `_edge_settings` + (`core/fingerprints.py:213-220`). It is *not* framing: `FramingSpec` + (`specs.py:244-298`) keeps `exclusions` exactly as landed and the post path + is untouched. +- **Same rectangles, oriented source pixels.** `SegmentationSpec.exclusions` + has the same type and the same validation as `FramingSpec.exclusions` + (`specs.py:291-298`, copied verbatim): a tuple of `CropSpec`, no bound + against the source, no count cap. They are mapped into the cut with the + existing `cut_exclusions` (`core/exclusion.py:40-52`) — verified in 7.4. +- **The switch, not the spec, is what the user edits.** The request builder + (`ui/request_builder.py:94-105`) fills both specs from the one tuple: + + ```python + segmentation=SegmentationSpec( + ..., exclusions=parameters.exclusions if parameters.exclusions_before_model else () + ), + framing=FramingSpec(..., exclusions=parameters.exclusions), + ``` + + With the switch on, every region is blanked before the model **and** still + zeroed after it. Pre-model is a superset of post-model, never an + alternative to it, so the request builder is one expression and the six + post sites (Decision 3) need no change. +- **The stored cut carries the blank.** `_produce_cut_frame` zeroes the + pre-model boxes in its result (7.4), so a pre-model cut set has alpha 0 + there *by construction*. That is what makes the two tuples independent: a + request whose `framing.exclusions` differs from the set's + `segmentation.exclusions` (the picker's Rebuild with a changed layout, + `ui/workspace_presentation.py:142-183`) is still correct, because nothing + the model produced inside a pre-model box survives in the stored bytes. No + cross-spec invariant, no validation in `RenderRequest.validate`. +- **Reducer.** One event, `ExclusionsBeforeModelChanged(enabled: bool)`, + reduced like `_reduce_trim` (`core/parameters.py:352-361`): reject a + non-`bool`, no-op on an equal value, and — like `_reduce_exclusions` + (`:383-402`) — **no preview invalidation when `cut_exclusions(exclusions, + crop)` is empty**, because the request is then byte-identical either way. + Otherwise `PreviewInvalidationReason.SEGMENTATION` (`core/tokens.py:26`). + `_reduce_exclusions` picks its reason by the switch: `SEGMENTATION` when it + is on, `CROP_CLEANUP` when it is off (`:398-402`). `_reduce_parameters_reset` + resets the flag and compares it in the `SEGMENTATION` branch (`:255-259`). +- **Persistence.** One QSettings bool `parameters/exclusions_before_model`, + parsed by the existing `_bool_value` (`parameters.py:569-577`) with its + per-key fallback; absent → `False`. `ui/preferences.py:15-27` gains the key, + `:44-72` persists it. The region string is unchanged. +- **Why not per-region (the fork).** A per-region flag is the more general + model and it is *not* chosen now, for three measured reasons: + 1. It is not the smaller diff. Two tuples through `ParameterState`, + `ParameterPresentation`, `CropPresentation`, preferences, the request + builder and the reducer, plus a selection identity that spans two tuples + in `ui/exclusion_canvas.py` (745 lines; the mode/select/drag code is + `:252-690`), plus a per-region checkable action in + `ui/exclusion_context_menu.py`. Estimated +200 lines and a split of the + canvas module. The global switch is ≈ +60 lines of `src/`. + 2. The coexistence case — a post-only chat box *and* a pre-model bystander + in one clip — is plausible and unmeasured. The maintainer's report names + one bystander. Ship the switch, measure whether mixed clips occur. + 3. It is upgradeable without a migration. The request-level shape + (`SegmentationSpec.exclusions`, the cut key, the manifest key, the + pipeline call, the fill, the tests in 7.8) is identical under a + per-region model; only the reducer/UI grain differs. A later + per-region flag reads the bool as "all regions flagged". + The alternative is recorded under *Rejected alternatives* with the two + shapes it could take. + +### 7.2 Cache and fingerprint — the non-empty rule, verified against the cut key + +`cut_cache_key_inputs` (`core/fingerprints.py:188-222`) gains one entry, +**present only when the effective boxes are non-empty**, and the preview +identity gains the same entry under its `segmentation` sub-object +(`:154-158`): + +```python +def _model_exclusions(request: RenderRequest) -> dict[str, object]: + boxes = cut_exclusions(request.segmentation.exclusions, request.crop) + if not boxes: + return {} + return { + "model_exclusions": { + "fill": MODEL_FILL_ID, # core/exclusion.py + "boxes": tuple((b.left, b.top, b.right, b.bottom) for b in boxes), + } + } +# cut_cache_key_inputs: {..., "edge_settings": _edge_settings(request), **_model_exclusions(request)} +# preview_fingerprint: "segmentation": {"model_id": ..., **_edge_settings(request), **_model_exclusions(request)} +``` + +**Why the fill identifier is inside the value.** The fill (7.3) is part of +what the model saw. If a later measurement changes it, the cut sets it must +invalidate are exactly the pre-model ones; bumping `PIPELINE_SCHEMA_VERSION` +(`fingerprints.py:30`) would orphan every set of every user. Carrying the id +inside the only-when-present value scopes the invalidation correctly. + +**Why tuples, not lists.** `_freeze_mapping` (`fingerprints.py:382-391`) +turns a list into a tuple but does not descend into it (`:387-388`), so a +list of lists would leave mutable inner lists behind the "recursively +immutable" docstring (`:197`). Tuples of ints freeze as-is, and +`_canonical_value` (`:394-399`) renders tuples as JSON arrays. + +**The key does not change for a user without pre-model regions — verified +case by case:** + +| Case | `segmentation.exclusions` | `cut_exclusions(…, crop)` | Entry | Key | +|---|---|---|---|---| +| Switch off (every 0.4.1 request) | `()` — request builder | `()` | absent | **byte-identical** to `tests/core/test_fingerprints.py:610-646`, which pins the canonical JSON literally | +| Switch on, no regions | `()` | `()` | absent | unchanged | +| Switch on, region entirely outside the crop | non-empty | `()` — guard at `exclusion.py:50` | absent | unchanged, and correctly so: the model input is pixel-identical | +| Switch on, region clipped to zero width or height by the crop | non-empty | `()` — same guard (`right > left and bottom > top`) | absent | unchanged | +| Switch on, region intersecting the crop | non-empty | boxes in cut space | present | changes → new cut set | + +- **Crop.** Boxes are crop-relative (`exclusion.py:46-49`), so a crop move + changes `"crop"` (`fingerprints.py:212`) and the boxes together. The same + region under two crops yields two keys, which is right: the model input + differs. +- **Rotation.** Regions and crop are both in oriented source pixels + (`specs.py:108-109`); rotation is decoded from the file and is covered by + `source_sha256`. The boxes are in cut space and carry no orientation, so + nothing further enters the key. +- **Manifest round trip.** `CutManifest` re-derives the key from its stored + inputs and refuses a mismatch (`workspace/_manifest.py:168-171`, `:295-301`). + `_freeze_json` accepts lists and tuples (`_manifest_validation.py:188`), + `_thaw_json` gives lists back (`:198-199`), and `_canonical_value` passes + lists through to `json.dumps`, so the hash of the thawed manifest inputs + equals the hash of the request's tuples. Verified by reading; pinned by + the round-trip test in 7.8. +- **The manifest validator must learn the key.** `_validate_cache_inputs` + (`_manifest_validation.py:76-90`) applies `_exact_keys` to + `cache_key_inputs`, so a manifest carrying `model_exclusions` is + `CUT_MANIFEST_INVALID` today — `find_matching_cut_workspace` would swallow + that (`render.py:592-599`) and re-segment on every render, and the set + would never be reusable. The validator gains an *optional* key: the exact + set is extended by `"model_exclusions"` when present, and a + `_validate_model_exclusions` helper checks `fill` with `_bounded_text` + (`:278`) and each box as four ints with `_bounded_int` (`:272`), + `0 <= left < right <= MAX_CUT_DIMENSION`, same for top/bottom. The module is + 344 lines, so the ratchet does not apply (`scripts/check_guardrails.py:120-128` + fires only above 800); the helper stays under 60 lines. **Not a baseline + question.** +- **Old binaries.** A 0.4.1 build reading a pre-model manifest gets + `CUT_MANIFEST_INVALID` from the strict schema and treats it as "no cut + set": it re-segments without the blank — degraded, not broken — and its + picker lists that set the way it lists any unreadable manifest today. + Sets without the key are untouched. Residual, accepted. +- **Two sets on disk.** Flipping the switch produces a new key; the old set + stays. Flipping back is a cache hit (`find_matching_cut_workspace`, + `render.py:571-600`, called at `:1049`) with no re-segmentation. The disk + cost is one extra set per clip, visible in the workspace dialog like any + other. +- **The picker's Rebuild.** `request_for_workspace` + (`ui/workspace_presentation.py:142-183`) rebuilds a request from a manifest + and must reproduce the key (`render.py:1224-1231` refuses a mismatch). It + restores `SegmentationSpec.exclusions` from `inputs["model_exclusions"]` by + translating each box back through the manifest crop: + `CropSpec(crop.x + left, crop.y + top, right - left, bottom - top)`. A + clipped box is inside the crop, so `cut_exclusions` maps it back onto + itself and the key matches. + +### 7.3 What the blank is filled with — measured, not guessed + +A hard black rectangle is an edge, and a segmentation network can read an +edge as an object boundary. If it does, the feature is worse than not using +it: the bystander is gone and a rectangle-shaped artefact takes its place. +**The fill is not chosen in this document.** It is chosen by the measurement +below, and the identifier of the winner is what `MODEL_FILL_ID` carries into +the key. + +**What the model actually sees** (`rembg/sessions/base.py:40-65`): the +frame is converted to RGB (`:49` — alpha is dropped), resized to the model's +input size (1024² for BiRefNet and ISNet, 320² for U²-Net; `birefnet_general.py:34-36`, +`u2net.py:32`, `dis_general_use.py:27`), scaled by its own maximum (`:52`), +then normalised per channel by `(x - mean) / std`. Twelve of the thirteen V1 +models use the ImageNet constants `(0.485, 0.456, 0.406) / (0.229, 0.224, 0.225)`; +`isnet-general-use` uses `(0.5, 0.5, 0.5) / (1, 1, 1)`. So a flat fill at the +model's *mean* is the input the network's first layer treats as zero — that +is the one candidate with a model-independent argument, and why it is the +fallback. It is still a rectangle. + +**Candidates**, each a pure function `(image: PIL RGBA, box: PixelBounds) -> None` +in `scripts/measure_model_fill.py`, the winner promoted to +`core/exclusion.py::blank_model_input`: + +| Id | Fill | Structure it introduces | +|---|---|---| +| `black` | `(0, 0, 0)` | a strong edge; normalises to ≈ −2.1 per channel | +| `mid-grey` | `(128, 128, 128)` | an edge; ISNet's zero | +| `imagenet-mean` | `(124, 116, 104)` | an edge; the zero for 12 of 13 models | +| `ring-mean` | the mean colour of the 4 px ring around the box, inside the frame | a softer edge; adapts to the local background | +| `blur` | the box contents under `GaussianBlur(radius = max(w, h) / 4)` | no edge, but a skin-coloured blob where a face was | +| `mirror` | the adjacent in-frame strip reflected into the box | no edge; **re-introduces whatever was next to the box**, including the subject's own edge | + +**Measurement.** Needs a real model; runs by hand; recorded in this +document, not in the test suite. + +- *Input.* Pairs of frames `(control, test)` at the same framing: `control` + has the wanted subject alone; `test` is `control` with a bystander + composited at the frame edge (a cutout of a second person from another + clip, pasted at two positions: touching the frame edge, and 40 px inside + it). At least three subjects × two bystander positions × two models + (`birefnet-portrait`, `u2net_human_seg`) = 12 pairs. The #80 clip has real + bystander frames and no control; it is used for the inside-box metric and + a visual check only, never for the decision. +- *Procedure.* For each pair and candidate `C`: fill the bystander box in + `test`, segment, take the alpha `A_C`. Also `A_none` (unfilled `test`) and + `A_ctrl` (`control`). Inference is deterministic on CPU, so the noise floor + is the spread *across pairs*, not across runs. +- *Compared.* Outside the box: `err_C = mean |A_C − A_ctrl|` and the IoU of + `A > 2 %` (the default threshold, `specs.py:248`) against `A_ctrl`. In a 16 + px band around the box: the same error, reported separately — this is where + "the rectangle edge became an object" shows up. Inside the box: `mean A_C`, + diagnostic only, because 7.4 zeroes it anyway. +- *Rejects a candidate.* `err_C > err_none` or band error above `err_none` + on **any** pair: the fill did more harm than the bystander it replaced. +- *Decides.* Among the survivors, the lowest mean `err_C`; the winner must + beat the runner-up by more than 1 IoU point on the majority of pairs. +- *Inconclusive.* Winner within 1 IoU point of the runner-up on most pairs, + or fewer than two survivors: **fallback `imagenet-mean`**, on the argument + above (the network's zero for 12 of 13 models, zero structure of its own). + `mirror` is never the fallback: it is the only candidate whose structure + depends on what stands beside the box. + +Whatever wins, `blank_model_input` is Pillow only (`image.paste`, at most +`ImageFilter.GaussianBlur` — both already in the tree) and `MODEL_FILL_ID` +names it. A different winner later is a one-line change in +`core/exclusion.py` that invalidates exactly the pre-model sets (7.2). + +### 7.4 Pipeline call site — inside `_produce_cut_frame`, two lines apart + +`_produce_cut_frame` (`jobs/render.py:1529-1608`) is the only +decode → crop → segment → cleanup path (`render.py:13`), shared by preview +(`:923`) and render (`:1104`). Both calls therefore see the blank without a +change of their own. + +- **The regions map with the existing `cut_exclusions`.** `apply_source_crop` + (`core/geometry.py:718-724`) returns `image.crop(...)` — the frame in cut + space with the crop's top-left at (0, 0); pre-model regions are oriented + source rectangles like the post ones; `cut_exclusions(regions, request.crop)` + (`exclusion.py:40-52`) is exactly the translation-and-clip both kinds need. + Verified: no second mapping. +- **Fill on the Pillow image, not the array.** `np.asarray(cropped)` at + `:1564` hands back a **read-only** view (measured on this branch: + `flags.writeable == False`, and `ascontiguousarray` returns it unchanged), + so writing into `input_frame` would raise. `cropped` is a fresh image from + `.crop()` and `paste` works in place — the same call `exclude_alpha` makes + (`exclusion.py:55-58`). So: + + ```python + cropped = apply_source_crop(decoded_image, crop_bounds) # :1558 + tracker.register(cropped) # :1559 + model_boxes = cut_exclusions(request.segmentation.exclusions, request.crop) + blank_model_input(cropped, model_boxes) # new + ... + input_frame = np.ascontiguousarray(np.asarray(cropped, dtype=np.uint8)) # :1564 + ``` + + The fill sits **after the crop and before the array conversion**, i.e. + before the model at `:1574`. No extra allocation, no tracker change. +- **Zero the box in the result.** rembg composites the *input* RGB with the + mask (`jobs/rembg_runtime.py:159-165`; `rembg/bg.py:296-314`), so the + filler pixels come back with whatever alpha the model gave them. After + `tracker.register(result)` at `:1606`: + + ```python + exclude_alpha(result, model_boxes) # new + ``` + + In place on the returned image; `result` is the only owner. This is what + guarantees 7.1's "stored cut carries the blank" and keeps filler RGB out of + every consumer, the preview's `_immutable_rgba(cut)` included (`:968`). +- **Relative to `_decontaminate_edge_colors_in_place`** (`:1611-1631`, called + at `:1600` on the model output): the fill runs before the model, the + decontamination on its output, the zeroing after both. Order between the + last two is immaterial — decontamination writes canonical black under + alpha 0 (`:1620`) and `exclude_alpha` writes `(0,0,0,0)` — but zeroing on + `result` rather than on `segmented` avoids touching the read-only array the + host hands back (`tests/jobs/render_support.py:114-117` models that). +- **Cost.** Three lines in a frozen module (2886, baseline `scripts/guardrails-baseline.json`). + `_produce_cut_frame` is already over the 60-line function budget, so the + `long_functions` count does not move (`check_guardrails.py:130-133`); the + module line count does, and that fails the ratchet. **This is a baseline + question for the maintainer**; the alternative — a wrapper module that + re-implements the pipeline to avoid the three lines — is rejected below. +- **The segmentation host is untouched.** It accepts any `H×W×3/4` uint8 + frame (`jobs/segmentation_host.py:605-621`) and never sees the regions. +- **The post path is not slower.** Nothing on it changed; with the switch + off `model_boxes` is `()` and both new calls are loops over nothing. + +### 7.5 What the user sees — the cost is named where it is paid + +Two rounds on this branch shipped controls that looked live and were not. +Everything below either dispatches a reducer event or is derived from state +the reducer already changed; nothing holds a checked state of its own. + +- **Inspector.** `ExclusionControls` (`ui/exclusion_controls.py`) gains a + `QCheckBox`, object name `exclusion_before_model`, text + `translate("Inspector", "Blank before the model")`, tooltip and accessible + description `translate("Inspector", "Re-segments the clip: the stored cut set is not reused")`. + `apply()` sets it from `presentation.exclusions_before_model` with signals + blocked, and enables it as `available and count > 0`, exactly like + `clear_button` (`:37`): a switch with no regions is inert, so it is + disabled rather than checked-and-idle. Toggling emits + `ExclusionsBeforeModelChanged(checked)` through the existing + `command_requested` signal (`:25`, wired at `inspector.py:312`); no + `main_window.py` change. +- **Count label.** Two literals, chosen by the flag, both Qt numerus forms: + `tr("%n region(s)", "", n)` as today (`:31-32`) and + `tr("%n region(s), blanked before the model", "", n)`. The literal is at + the call site; the flag selects which call runs. +- **Canvas.** `_paint_exclusions` (`ui/exclusion_canvas.py:174-221`) keeps + its colours and, when `presentation.exclusions_before_model`, draws the + effective rectangle with a `Qt.BrushStyle.BDiagPattern` brush in the same + `#E5484D` over the translucent fill and a 2 px solid pen instead of 1 px. + Hatched = the model never sees it; plain = removed afterwards. Not + colour-only. `CropPresentation` gains `exclusions_before_model: bool = False` + (`ui/crop_presentation.py:21`, filled at `:64`). +- **Announcement.** `_announce_crop` (`exclusion_canvas.py:699-745`) appends + `translate("CropCanvas", "; blanked before the model")` when the flag is + set — the same append pattern the selected-region readout already uses + (`exclusion_controls.py:57-59`, `:69-71`). No variable reaches `translate`. +- **The cost is shown by the stale-preview banner, for free.** The reducer + invalidates with `SEGMENTATION` (7.1), which the presentation model already + words as "Segmentation" (`ui/presentation_model.py:14`) — the same banner + a model change produces, which is the honest comparison: the next Preview + runs the model, the next Render segments the clip. With the switch on, a + region drag also invalidates as `SEGMENTATION`, so the user sees at the + first nudge that regions are now a segmentation setting. +- **Nothing new in the job dialog.** A render with a new key is an ordinary + render; the progress stages already say "Cut frame n of m" + (`render.py:1129`). +- **New strings** (all literals, German entries required): + `"Blank before the model"`, `"Re-segments the clip: the stored cut set is not reused"`, + `"%n region(s), blanked before the model"`, `"; blanked before the model"`. +- **README.** The bullet at `README.md:29-30` ("made transparent after + segmentation") gains one sentence on the switch; `main-window.png` is + regenerated with `scripts/screenshots.py:127`'s region and the switch on, + so the hatch is visible. + +### 7.6 Degradation — the post path's empty-mask handling carries over + +- **A pre-model region covers the whole frame.** The model sees a flat + fill. BiRefNet then min-max-normalises a near-constant prediction + (`birefnet_general.py:41-44`), which can produce NaN and an arbitrary + mask; alpha matting raises on an empty trimap and rembg falls back to the + naive cutout (`rembg/bg.py:298-309`). None of that reaches the output: + 7.4 zeroes the box in the result, and a whole-frame box zeroes the whole + frame. Every cut frame is fully transparent → the union is empty → + Decision 5 applies unchanged: trim is skipped, the untrimmed canvas is + encoded, `notes` records it (`render.py:1378-1381`), the UI facts worker + degrades through `union_alpha_bounds_or_none`. **Carries over; nothing + new.** +- **A pre-model region covers the whole subject.** Same path; what the + model does with the filler's edge inside the box is zeroed, and what it + does *outside* is exactly the band metric in 7.3. +- **A pre-model region entirely outside the crop.** No box, no key change, + no re-segmentation — the 7.2 table. +- **Switch on, no regions.** Inert control (7.5), no invalidation (7.1). +- **Switch flipped while a job runs.** `ExclusionsBeforeModelChanged` is a + `ParameterEvent` and goes through the same `can_edit` gate as the others + (`parameters.py:191`); the controls are disabled. +- **A persisted `True` from a build that knew the switch, read by one that + does not.** Ignored — `load_parameters` reads only `_KEYS` + (`preferences.py:30-34`). Regions still apply post. Degraded, honest. +- **Nothing refuses.** No new `ValidationError` on a user-reachable path. + +### 7.7 File-by-file change list + +Estimates; **bold** = frozen in `scripts/guardrails-baseline.json` and over +800 lines, so growth fails the ratchet. + +| File | Now | Δ | Notes | +|---|---|---|---| +| `src/matteloop/core/exclusion.py` | 97 | +22 | `MODEL_FILL_ID`, `blank_model_input(image, boxes)`, docstring: the model-input site | +| `src/matteloop/core/specs.py` | 706 | +9 | `SegmentationSpec.exclusions` + the validation block copied from `FramingSpec` | +| `src/matteloop/core/fingerprints.py` | 484 | +14 | `_model_exclusions(request)`; spread into `cut_cache_key_inputs` and `preview_fingerprint`'s `segmentation` | +| `src/matteloop/core/parameters.py` | 638 | +34 | field, `ExclusionsBeforeModelChanged`, `_reduce_exclusions_before_model`, reason selection in `_reduce_exclusions`, reset, `parameters_from_values` | +| `src/matteloop/jobs/workspace/_manifest_validation.py` | 344 | +16 | optional `model_exclusions` in `_validate_cache_inputs` + `_validate_model_exclusions`. Under 800: not a baseline question | +| **`src/matteloop/jobs/render.py`** | 2886 | **+3** | `_produce_cut_frame`: `model_boxes`, `blank_model_input`, `exclude_alpha(result, …)`. **Flag: needs `--update` with agreement.** | +| `src/matteloop/ui/request_builder.py` | 112 | +1 | `exclusions=` on `SegmentationSpec` | +| `src/matteloop/ui/workspace_presentation.py` | 214 | +8 | restore `exclusions` from `model_exclusions` in `request_for_workspace` | +| `src/matteloop/ui/preferences.py` | 91 | +2 | key + persist | +| `src/matteloop/ui/parameter_presentation.py` | 85 | +2 | field + mapping | +| `src/matteloop/ui/crop_presentation.py` | 63 | +2 | field + mapping | +| `src/matteloop/ui/inspector_reset.py` | — | +1 | compare the flag | +| `src/matteloop/ui/exclusion_controls.py` | ~95 (in flux) | +18 | checkbox, `apply`, `tab_widgets`, count-label variant | +| `src/matteloop/ui/exclusion_canvas.py` | 745 | +12 | hatch in `_paint_exclusions`, suffix in `_announce_crop` → **~757 of 800. Tight; nothing else may land here without a split** | +| **`src/matteloop/ui/inspector.py`** | 904 | 0 | the checkbox lives inside `ExclusionControls`; `tab_widgets()` already spreads (`:834`, `:855`) | +| `src/matteloop/ui/main_window.py` | 357 | 0 | `command_requested` already routed | +| `resources/matteloop_{en,de}.ts`, `.qm` | — | regen | four new strings, German by hand | +| `scripts/measure_model_fill.py` | new | +90 | the 7.3 harness: six candidates, the pair loop, the table. Run by hand, not a test | +| `scripts/screenshots.py` | — | +1 | switch on in `_state` | +| `README.md`, `assets/screenshots/main-window.png` | — | +1, regen | | +| `docs/v1-scope.md` | — | +3 | extend the exclusion-regions row: pre-model option, keyed in the cut. **First** (G8) | +| `scripts/guardrails-baseline.json` | — | 1 entry | `jobs/render.py`, only after agreement | +| `tests/jobs/render_support.py` | — | +2 | `FakeSegmenter` keeps `frames.append(frame.copy())` so a test can see what the model saw (`:119-128` records only requests today) | + +**One baseline question for the maintainer:** `jobs/render.py` +3. Everything +else is under budget. Nothing touches `core/state.py`, `core/geometry.py`, +`jobs/segmentation_host.py` or `jobs/transform_stage.py`. + +### 7.8 Test list + +One test per behaviour, named after it (G7). "Fails without" names the +missing piece. **unit** = cheap and Qt-free; **Qt** = offscreen widgets; +**model** = a real rembg session, run by hand. + +| Test | Fails without | Kind | +|---|---|---| +| `tests/core/test_specs.py::test_segmentation_spec_rejects_exclusions_that_are_not_crop_specs` | the validation block | unit | +| `tests/core/test_exclusion.py::test_blank_model_input_fills_each_box_and_nothing_else` — every RGB inside is `MODEL_FILL_ID`'s value, alpha untouched, outside byte-identical | `blank_model_input` | unit | +| `…::test_blank_model_input_with_no_boxes_leaves_the_image_byte_identical` | the empty loop (post-path cost) | unit | +| `tests/core/test_fingerprints.py::test_cut_key_without_model_exclusions_matches_the_canonical_schema` — extends `:610-646`: switch semantics with `segmentation.exclusions=()` produce the pinned JSON, and `"model_exclusions"` is absent from `cut_cache_key_inputs` | the non-empty rule; failing it orphans every 0.4.1 cut set | unit | +| `…::test_cut_key_ignores_a_model_exclusion_outside_the_crop_or_clipped_to_nothing` | the `cut_exclusions` guard feeding the key | unit | +| `…::test_cut_key_tracks_model_exclusion_boxes_and_the_fill_identifier` — two regions, two keys; monkeypatched `MODEL_FILL_ID` changes the key | `_model_exclusions` | unit | +| `…::test_cut_key_inputs_with_model_exclusions_are_recursively_immutable` | tuples, not lists, in the payload | unit | +| `…::test_preview_identity_tracks_model_exclusions` | the `segmentation` spread | unit | +| `…::test_render_and_union_identities_ignore_model_exclusions` — they depend on `cut_key` only | that nothing was added to `_framing` / `cut_union_fingerprint` | unit | +| `tests/jobs/test_workspace.py::test_manifest_accepts_model_exclusions_and_reproduces_the_cut_key` — `CutManifest.create` with the new inputs, JSON round trip, `cache_key_for` equals `cut_cache_key` | the validator's optional key and the list/tuple round trip | unit | +| `…::test_manifest_rejects_malformed_model_exclusion_boxes` — inverted box, negative, non-int | `_validate_model_exclusions` | unit | +| `…::test_manifest_without_model_exclusions_still_validates` — a 0.4.1 payload verbatim | that the key stayed optional | unit | +| `tests/core/test_parameters.py::test_exclusions_before_model_invalidates_the_preview_as_segmentation` | the reducer | unit | +| `…::test_exclusions_before_model_with_no_effective_region_changes_state_without_invalidating` | the empty-box no-op | unit | +| `…::test_region_change_invalidates_as_segmentation_while_the_switch_is_on` | reason selection in `_reduce_exclusions` | unit | +| `…::test_parameters_reset_clears_exclusions_before_model` | reset | unit | +| `…::test_exclusions_before_model_settings_value_round_trips_and_a_malformed_value_falls_back_to_false` | parse | unit | +| `tests/jobs/test_render.py::test_render_blanks_model_exclusions_in_the_model_input_and_zeroes_them_in_the_stored_cut` — `FakeSegmenter.frames[i]` carries the fill inside the box and the original outside; the stored PNG is `(0,0,0,0)` there | the two `_produce_cut_frame` lines; masking only after the model passes the second assertion and fails the first | unit | +| `…::test_render_with_model_exclusions_reads_the_original_pixels_outside_the_box` | that the fill did not leak (e.g. a ring or blur candidate overrunning the box) | unit | +| `…::test_render_with_the_switch_off_sends_the_model_the_unblanked_frame` | the post path's cost contract | unit | +| `…::test_render_reuses_the_unblanked_cut_set_after_the_switch_is_turned_off` — render, flip, render, flip, render: the third finds the first set | 7.2's two-sets claim | unit | +| `tests/jobs/test_render.py::test_render_with_a_whole_frame_model_exclusion_encodes_the_untrimmed_canvas_and_notes_it` | Decision 5 carrying over; a NaN mask from a flat input must not surface | unit | +| `tests/jobs/test_preview.py::test_preview_shows_no_filler_pixels_inside_a_model_exclusion` — `PreviewResult.cut` is `(0,0,0,0)` in the box | `exclude_alpha(result, …)` | unit | +| `tests/jobs/test_rebuild.py::test_rebuild_of_a_model_exclusion_set_reproduces_its_key_from_the_manifest` — through `request_for_workspace` | the reverse mapping in `workspace_presentation.py` | unit | +| `tests/ui/test_parameter_inspector.py::test_before_model_checkbox_dispatches_and_is_inert_without_regions` — dispatch count 1 per toggle; disabled at count 0; reflects state, never holds it | the checkbox wiring; a stayed-checked control scores 0 dispatches | Qt | +| `…::test_before_model_count_label_names_the_blank` | the second literal | Qt | +| `tests/ui/test_crop_canvas.py::test_model_exclusions_are_painted_hatched_and_announced` — sample a pixel of the pattern; accessible description ends with the suffix | canvas + announcement | Qt | +| `tests/ui/test_parameter_persistence.py::test_exclusions_before_model_persists_as_a_bool` | preferences | Qt | +| `tests/ui/test_parameter_requests.py` (+2 assertions) — switch on fills `segmentation.exclusions`, off leaves `()`, `framing.exclusions` both times | request builder | Qt | +| `tests/test_translations.py::test_catalogues_match_lupdate_extraction` (existing) | German entries | Qt | +| `scripts/measure_model_fill.py` on the 12 pairs of 7.3, table pasted into this document under 7.3 | the fill decision — **must run before `MODEL_FILL_ID` is set** | model | +| A real render of the #80 clip with the switch on and off, compared by eye | the reason this exists | model | + +### 7.9 Implementation order + +0. `docs/v1-scope.md` row (G8), and the one baseline question answered. +1. **The measurement first**, because its result is a constant every later + step reads: `scripts/measure_model_fill.py`, the pairs, the table into + 7.3, `MODEL_FILL_ID` decided. If it is inconclusive, `imagenet-mean`, and + the table still goes into the document. +2. `core/exclusion.py`, `specs.py`, `fingerprints.py`, + `workspace/_manifest_validation.py` + their tests. +3. `render.py`'s three lines, `render_support.py`, the jobs tests. **Measure + the delta; stop for the maintainer if the baseline is not yet agreed.** +4. `parameters.py`, `preferences.py`, `request_builder.py`, + `workspace_presentation.py`, the presentations + tests. +5. `exclusion_controls.py`, `exclusion_canvas.py`, catalogues + Qt tests. + **Check `exclusion_canvas.py` against 800 here.** +6. Screenshot, README. + +Each step verified with the standard command from `CLAUDE.md`. + + +## Not in scope + +- Numeric x/y/w/h fields per region, region naming, a region list widget. +- A UI channel for `notes` (the trim-skipped message reaches the log). +- Reset of regions on source change. They are carried and **clipped** + (Decision 2b). +- A cap on the number of regions. +- Merging overlapping regions into a canonical cover (Decision 4, false miss 2). +- Caching an empty union (Decision 4, false miss 3). +- Retaining the unmasked preview cut so a region change re-frames without the + model (*Constraints*). +- Touch hit-testing beyond what `InteractionGeometry.hit_test(touch=…)` + already gives the selected region. +- Anything in `core/geometry.py`, `core/state.py`. +- A per-region pre/post flag (Decision 7.1 chooses one global switch). +- Choosing the model-input fill by argument; it is chosen by the + measurement in Decision 7.3. +- Retaining an unblanked cut set's frames for a pre-model set, or any + migration between the two (they are two keys, both kept on disk). + +## Failure modes considered + +- **Region entirely inside the crop's excluded margin** — no box; the reducer + no-ops on unchanged effective boxes, so nothing stales; the rectangle is + still painted, and the pixels under it are gone anyway because the crop + removed them. +- **Carried region partly outside a smaller new source** — clipped on load to + its intersection, so what is painted is what is masked. +- **Carried region entirely outside a smaller new source** — dropped on load. +- **Anamorphic source** — correct only after the Decision 2a prerequisite + lands; regions must not be implemented before it. +- **A drag across 200 pointer samples** — one dispatch, one facts worker. +- **Corrupt or mismatched stored frames while regions exist** — still fails + loudly; only a genuinely empty mask degrades (Decision 5). +- **Same regions, different order** — identical fingerprint and bytes + (`cut_exclusions` sorts and de-duplicates). +- **Regions during a running job** — `ExclusionsChanged` goes through the + existing `can_edit` gate (`parameters.py:181-182`); the controls disable + and region mode is left. +- **Persisted string malformed** — `()`, per-key fallback. +- **Region mode with no source** — controls disabled; `set_exclusion_edit` + with `presentation is None` paints nothing. +- **External edit of a stored frame** — the union cache is already dropped + (`_cut_ops.py:380`); exclusions are applied on read, so edited pixels inside + a box are zeroed like any other. +- **Pre-model regions** — whole frame, whole subject, outside the crop, + switch without regions, an old binary reading a new manifest: Decision 7.6. +- **`render.py` growth** — see the change list. The first draft's "≈0" was a + guess; this revision budgets +5…+15 and makes measuring it a gate in the + implementation order, not a hope. + +## File-by-file change list + +**Landed** with Decisions 1–6; kept as the record of that work. The +pre-model addition has its own list in Decision 7.7. + +Line deltas are estimates; **bold** = frozen in the baseline. + +**Prerequisite, lands separately** (Decision 2a — its own issue, branch and +`fix:` commit, before any of the below): + +| File | Now | Δ | Notes | +|---|---|---|---| +| `src/matteloop/core/crop.py` | 291 | ~0 | `_orientation_transform` viewport: oriented display size for every rotation | +| `tests/core/test_crop.py` | — | +25 | rotation × PAR matrix; the current test is one passing cell of it | + +**This feature:** + +| File | Now | Δ | Notes | +|---|---|---|---| +| `src/matteloop/core/exclusion.py` | new | +95 | `cut_exclusions`, `exclude_alpha`, `union_alpha_bounds_or_none`, docstring enumerating the three producers and six sites | +| `src/matteloop/core/specs.py` | 697 | +14 | `FramingSpec.exclusions` + validation | +| `src/matteloop/core/fingerprints.py` | 465 | +25 | `cut_union_fingerprint`, `_framing(request)` with the non-empty rule | +| `src/matteloop/core/parameters.py` | 553 | +60 | field, `ExclusionsChanged`, reducer comparing **effective** boxes, `clip_exclusions`, reset, settings parse | +| **`src/matteloop/core/state.py`** | 882 | +1 | `clip_exclusions` on `SourceLoaded`. **Flag: frozen; needs `--update` with agreement** | +| `src/matteloop/jobs/transform_stage.py` | 177 | +30 | `cached_union`, degrade in `framing_plan`, `exclusions` through `stage_encoder_frames`/`_stage_frame` | +| **`src/matteloop/jobs/render.py`** | 2886 | **+5 … +15** | sites 1–4 (site 2 is a reorder, not an addition); `_matching_union` and `_rebuild_union` collapse onto `cached_union`; `union_alpha_bounds_or_none` at site 3. **Flag: frozen, and the first draft's "≈0" was not a safe estimate. Measure at step 3 and ask before touching the baseline.** | +| `src/matteloop/ui/preferences.py` | 91 | +6 | key + persist | +| `src/matteloop/ui/request_builder.py` | 112 | +1 | pass `exclusions` | +| `src/matteloop/ui/parameter_presentation.py` | 85 | +2 | field + mapping | +| `src/matteloop/ui/crop_presentation.py` | 63 | +3 | field + mapping | +| `src/matteloop/ui/transform_stage.py` | 721 | **+45** | `_crop_from_manifest`, `cached_union`, `union_alpha_bounds_or_none` branch, boxes through `_iter_stored_frames` and `FrameLoadWorker` — lands at **~766 of 800**. **Flag: tight. If it crosses, split `_CutFactsWorker` into a sibling module rather than raising the budget.** | +| `src/matteloop/ui/result_player.py` | 544 | +8 | boxes through `FrameLoadWorker`/`_load_one_frame` | +| `src/matteloop/ui/crop_canvas.py` | 359 | **+185** | mode, painting, rubber band, select/move/delete, gesture-local state and the release commit, announcement → **~544**. Over half the budget in one file; if it passes ~600, split the region mode into `ui/exclusion_canvas.py` as a `CropCanvas` subclass, the way `ResultPlayerCanvas` already is | +| `src/matteloop/ui/exclusion_controls.py` | new | +80 | the three controls, `apply`, `tab_widgets` | +| **`src/matteloop/ui/inspector.py`** | 897 | +5 | construct, add row, apply, tab order. **Flag: frozen; needs `--update` with agreement** | +| `src/matteloop/ui/main_window.py` | 351 | +1 | connect `edit_toggled`. **Not a baseline question** — the ratchet applies only above 800 lines (`scripts/check_guardrails.py:124-126`) | +| `src/matteloop/ui/copy.py` | 677 | +1 | `"Exclusions"` in `inspector_label` | +| `resources/matteloop_{en,de}.ts`, `.qm` | — | regen | `pyside6-lupdate`/`lrelease` per `docs/building.md:105-111`; German entries by hand | +| `scripts/screenshots.py` | — | +3 | one demo region | +| `README.md`, `assets/screenshots/main-window.png` | — | +6, regen | | +| `docs/v1-scope.md` | — | +4 | **first**, not last (G8) | +| `scripts/guardrails-baseline.json` | — | 2 entries | `core/state.py`, `ui/inspector.py`; only after agreement | + +**Two baseline questions for the maintainer, both +1 … +5 lines:** +`core/state.py` (the clip that makes painted == masked a property) and +`ui/inspector.py` (hosting the controls row). `jobs/render.py` is a third only +if it measures positive after the collapse. + +## Test plan + +**Landed** with Decisions 1–6; Decision 7.8 lists the pre-model tests. + +Named after behaviour (G7). "Fails without" names the missing piece. + +**Cheap, Qt-free** + +| Test | Fails without | +|---|---| +| `tests/core/test_crop.py::test_oriented_rect_round_trips_at_every_rotation_and_pixel_aspect` — rotation {0,90,180,270} × PAR {1, 2, 1.5, 0.75}, asserting the painted rect equals the oriented rect's position in `content_rect` | **Decision 2a. Fails on `main` today in 6 of 16 cells.** Belongs to the prerequisite commit | +| `tests/core/test_exclusion.py::test_cut_exclusions_translates_by_the_crop_origin_and_clips_to_the_crop` | the mapping | +| `…::test_cut_exclusions_drops_a_region_outside_the_crop` | the empty guard | +| `…::test_cut_exclusions_is_order_independent_and_deduplicated` | sorting/set | +| `…::test_exclude_alpha_zeroes_rgba_inside_each_box_and_nothing_else` | `exclude_alpha` | +| `tests/core/test_specs.py::test_framing_spec_rejects_exclusions_that_are_not_crop_specs` | validation | +| `tests/core/test_fingerprints.py::test_render_and_preview_identity_track_effective_exclusions_only` | `_framing(request)`; a region outside the crop must **not** change it | +| `…::test_union_identity_tracks_effective_exclusions` | `cut_union_fingerprint` | +| `…::test_identities_without_exclusions_match_the_previous_schema` — asserts the canonical payload has no `"exclusions"` key when empty | the non-empty rule; failing it orphans every v0.4.1 union cache | +| `tests/core/test_parameters.py::test_exclusions_changed_invalidates_the_preview_as_crop_cleanup` | the reducer | +| `…::test_moving_a_region_outside_the_crop_does_not_invalidate_the_preview` | the effective-box comparison (Decision 4, false miss 1) | +| `…::test_source_load_clips_carried_regions_and_drops_those_outside` | `clip_exclusions` (Decision 2b) | +| `tests/core/test_state.py::test_loading_a_smaller_source_leaves_every_region_inside_the_frame` | the `SourceLoaded` call site — the invariant painted == masked rests on | +| `…::test_parameters_reset_clears_exclusions` | reset | +| `…::test_exclusions_settings_string_round_trips_and_a_malformed_entry_falls_back_to_none` | parse | + +**Jobs, existing fakes in `tests/jobs/render_support.py`** + +| Test | Fails without | +|---|---| +| `tests/jobs/test_preview.py::test_preview_zeroes_excluded_alpha_in_the_cut_image_and_its_bounds` | site 1 | +| `tests/jobs/test_render.py::test_render_stores_unexcluded_cuts_but_unions_and_encodes_without_the_region` — stored PNG keeps alpha; output frame alpha 0 in the box; `union_metadata.bounds` excludes it | sites 2 and 4, **and the stage-before-mask order at site 2**: masking first would write masked PNGs and the assertion on the stored file fails | +| `…::test_render_peak_rgba_owner_count_is_unchanged_by_exclusions` | the no-copy order; a `cut.copy()` at site 2 raises the reported peak | +| `tests/jobs/test_rebuild.py::test_rebuild_with_a_region_misses_the_cached_union_and_recomputes_it` | site 3 + fingerprint | +| `…::test_rebuild_without_regions_reuses_union_metadata_written_before_exclusions_existed` — metadata hand-built with the v0.4.1 payload | the non-empty rule | +| `tests/jobs/test_transform_stage.py::test_framing_plan_skips_trim_without_a_union` (replaces `:34`) | Decision 5 | +| `…::test_stage_encoder_frames_zeroes_excluded_boxes_before_framing` | site 4 | +| `tests/jobs/test_render.py::test_render_with_trim_and_an_empty_mask_encodes_the_untrimmed_canvas_and_notes_it` (replaces the abort half of `:1525-1552`) | Decision 5 | +| `…::test_render_still_refuses_a_trimmed_box_below_the_minimum_dimension` — the `visible_size=10` branch at `:1522`, kept verbatim | that this degradation did **not** swallow the minimum-dimension refusal | +| `tests/core/test_exclusion.py::test_union_alpha_bounds_or_none_returns_none_for_an_empty_mask` | the empty signal | +| `…::test_union_alpha_bounds_or_none_still_raises_for_mismatched_canvases_and_non_rgba_frames` | **review finding 4** — a blanket `INVALID_FRAMING` catch passes the first of these two and fails this one | + +**Qt, offscreen** + +| Test | Fails without | +|---|---| +| `tests/ui/test_transform_stage.py::test_facts_ignore_cached_union_metadata_whose_fingerprint_does_not_match` | `cached_union` in the UI — **fails on `main` today** | +| `…::test_region_change_recomputes_facts_and_reloads_player_frames_without_the_region` | sites 5 and 6 | +| `…::test_empty_union_degrades_to_the_untrimmed_framed_size` | the `None` branch in `_resolve_union` | +| `…::test_mismatched_stored_frames_still_fail_the_facts_worker` | **review finding 4** — passes on a blanket catch, which is the point | +| `tests/ui/test_crop_canvas.py::test_a_region_drag_dispatches_exactly_once_on_release` — press, 20 moves, release; count dispatches | **review finding 3**; a per-move dispatch scores 21 | +| `…::test_an_abandoned_rubber_band_dispatches_nothing` | the release guard | +| `tests/ui/test_crop_canvas.py::test_region_mode_rubber_band_adds_a_region_on_release` | rubber band | +| `…::test_region_mode_discards_a_rubber_band_below_eight_pixels` | the threshold | +| `…::test_region_mode_gives_the_crop_no_hit_targets` — press on a crop handle under a region starts nothing for the crop | the mode switch | +| `…::test_click_selects_a_region_and_the_same_drag_moves_it` | press rule 2 | +| `…::test_delete_removes_the_selected_region` | keys | +| `…::test_leaving_region_mode_restores_crop_handles_and_keeps_painting_regions` | mode exit | +| `…::test_region_mode_announces_the_selected_region_bounds` | accessibility | +| `tests/ui/test_parameter_inspector.py::test_exclusion_controls_toggle_edit_mode_and_clear_regions` | controls | +| `tests/ui/test_parameter_persistence.py::test_exclusions_persist_as_one_settings_string` | preferences | +| `tests/ui/test_parameter_requests.py` (+1 assertion) | request builder | +| `tests/test_translations.py::test_catalogues_match_lupdate_extraction` (existing) | German entries | + +Screenshot regeneration and a real render on the #80 clip are manual. + +## Implementation order + +**Landed** for Decisions 1–6; Decision 7.9 gives the order for the +pre-model addition. + +**0. `docs/v1-scope.md` first.** G8 (`docs/engineering-guardrails.md:244`) +makes that file authoritative and forbids implementing an out-of-scope design +requirement, so the scope entry is step zero, not a tidy-up at the end. Nothing +below starts until it is recorded and the two baseline questions +(`core/state.py`, `ui/inspector.py`) have the maintainer's answer. + +**1. The prerequisite, on its own branch and its own issue.** The +`_orientation_transform` viewport fix in `core/crop.py` plus the rotation × PAR +matrix (Decision 2a). It is a `fix:` with a `Trigger:` line naming the +anamorphic repro, it stands alone, and it must not be mixed into a feature +commit. + +Then, on this branch: + +2. `core/exclusion.py`, `specs.py`, `fingerprints.py` + their tests. +3. `jobs/transform_stage.py` (`cached_union`, degrade, encoder boxes) + + `render.py` sites 1–4 + jobs tests. **Measure `render.py`'s net delta here + and stop for the maintainer if it is positive.** +4. `parameters.py` (including `clip_exclusions` and the effective-box + comparison), the `core/state.py` line, `preferences.py`, + `request_builder.py`, `parameter_presentation.py`, `crop_presentation.py` + + tests. +5. `ui/transform_stage.py`, `result_player.py` (sites 5, 6) + tests. **Check + the line count against the 800 budget here**, and split rather than raise + it. +6. `crop_canvas.py` region mode and the gesture contract + tests. **Check the + line count here too.** +7. `exclusion_controls.py`, the five inspector lines, the main-window line, + `copy.py`, catalogues. +8. Screenshots, README. + +Each step is verified with the standard command from `CLAUDE.md`. + +## Rejected alternatives + +- **Regions in cut space.** Moving the crop would drag the overlay mask with + it; the overlay is fixed in the source frame. +- **A pre-segmentation mask *instead of* the post mask.** Re-segments on + every drag (crop is in the cut key) and is model-dependent. The post path + stays the default; the pre-model path is an *addition* (Decision 7), not a + replacement. +- **Zeroing inside `read_cut` / `FrameReader.read`.** Would corrupt the + identity consumers (hash scans, external-edit detection, CAS) that share the + readers and must see the stored bytes. +- **Exclusions on `FramingPlan`, applied inside `apply_framing`.** Covers the + two apply sites but none of the four measure sites, and grows the frozen + `geometry.py`. The seam would be no narrower. +- **Baking exclusions into the stored cut frames.** Makes the cut set depend + on framing; a region change would re-segment or orphan the set. +- **Always emitting `"exclusions": []` in fingerprints.** Cleaner JSON, but it + turns every existing union cache and every existing preview/render + fingerprint into a miss on upgrade for users who never draw a region. +- **A new manifest key for the exclusions.** The parser refuses unknown keys + (`_manifest.py:361-365`) and the workspace modules are frozen; the + fingerprint field already carries the identity. +- **Raw (unclipped) regions in the fingerprint.** A region outside the crop + would stale the preview and the render for a pixel-identical result. +- **An `ExclusionRegion` type.** `CropSpec` is the same rectangle with the + same validation; a second type would need its own drag/nudge/clamp helpers. +- **Extending `build_crop_geometry`'s priority to N regions.** N×9 targets, an + overlap policy, and growth in frozen `geometry.py`; the mode switch makes + the overlap question disappear. +- **Reducer state for the edit mode.** `AppState` is in frozen `state.py`; + `TransformGroup.crop_edit_toggled` already shows UI-local mode works. +- **Emitting the rubber band live.** Would put half-drawn regions through the + reducer and stale the preview per mouse move; a second dispatch would be + needed to discard a tiny one. One dispatch on release is smaller. +- **Resetting regions on source load.** Loses the case the issue describes — + the same stream layout across clips. Clipping costs the same one line in + `state.py` and keeps it. +- **A new oriented→widget mapping just for regions** (the review's + prescription for finding 1). It would leave the crop mispainted on + anamorphic sources, and put two transforms where the codebase has one. The + root cause is one wrong expression; fix that instead. +- **Folding the `core/crop.py` fix into this feature.** It has its own trigger + and its own repro and predates this work; a reviewer must be able to see it + alone. Scoped out as a prerequisite (Decision 2a). +- **Clipping carried regions only for display**, leaving the spec unclipped. + Avoids the frozen `state.py` line, but the first drag of a clipped region + would snap it to the clipped size — a visible jump — and the fingerprint + would hash a rectangle the user cannot see. +- **Using `clamp_crop` to fit carried regions.** It slides to preserve size + (deliberately, for issue #25); sliding an exclusion moves it onto wanted + content. +- **Catching `INVALID_FRAMING` to detect an empty mask** (the first draft). + The code also means non-RGBA, mismatched canvases and a plan mismatch, so + the catch would present a corrupt cut set as a working one (Decision 5). +- **Masking a `cut.copy()` in the render loop** (the first draft). Staging + before masking removes the copy, the extra full-resolution allocation and + the ownership-tracker question together. +- **Dispatching region drags live, like the crop.** Measured cost: one QThread + and ~12 QSettings writes per pointer sample, on a path with no debounce + (Decision 6). +- **Merging overlapping regions into a canonical cover.** Would close false + miss 2, but it is real geometry code for a cache miss nobody has measured. +- **Retaining the unmasked preview cut to re-frame locally.** Would make a + region change free for the preview, but it is a preview-caching feature in + its own right and touches frozen `state.py` far more than one line. +- **Keeping the empty-union refusal.** Zero lines, and the user can fix it — + but the untrimmed canvas is exactly what "exclude everything" means, and + the render must not abort for it. Recorded as the fork; degrade chosen. +- **A region count cap.** Nothing measured needs one; clipping bounds the + cost per region to one `paste`. +- **A per-region pre/post flag** (Decision 7.1). Either a second tuple + (`ParameterState.model_exclusions`) plumbed through every presentation, + the preferences, the reducer and a two-tuple selection identity in the + 745-line canvas, or an `ExclusionRegion(CropSpec)` subclass carrying + `before_model` that every drag helper would have to re-wrap because + `crop_from_drag`/`nudge_crop` construct plain `CropSpec`s. Both are larger + than the switch, the mixed-clip case is unmeasured, and the request-level + shape is identical, so the switch can grow into either later. +- **Pre-model boxes in cut space on `SegmentationSpec`.** Would save the + reverse mapping in `request_for_workspace`, but puts a crop-dependent + rectangle type on a spec that is otherwise crop-independent, and the + regions the user draws are source rectangles; `cut_exclusions` already + does the one mapping for both kinds. +- **Requiring `segmentation.exclusions ⊆ framing.exclusions` in + `RenderRequest.validate`.** Unnecessary once `_produce_cut_frame` zeroes + the pre-model boxes in its result (7.4): the stored set is correct under + any framing, and the picker's Rebuild needs no union of the two tuples. +- **Keeping the filler RGB in the cut and relying on the post mask.** The + picker can rebuild a pre-model set with a different post layout, and the + filler would then show through with whatever alpha the model gave it. + Zeroing costs one `exclude_alpha` in the producer. +- **The fill identifier in `PIPELINE_SCHEMA_VERSION`.** Invalidates every + cut set of every user for a change that touches only pre-model sets. +- **Always emitting `"model_exclusions"` in the cut key.** Same rejection as + the framing case: it invalidates every 0.4.1 cut set on update. +- **Filling the numpy array instead of the Pillow image.** `np.asarray` of a + Pillow image is read-only (measured); a writable copy is one crop-sized + allocation per frame and an ownership-tracker entry, for nothing. +- **A wrapper module around `_produce_cut_frame` to keep `render.py` at + 2886.** The pipeline is one function on purpose (`render.py:13`); a second + copy of decode → crop → segment for three lines is the spiral the + guardrails describe. +- **Declaring a fill without measuring.** A hard rectangle may read as an + object edge to the model; if it does, the feature is worse than not + using it. The measurement in 7.3 decides, and `imagenet-mean` is the + fallback only because it is the network's own zero for 12 of 13 models. +- **`mirror` as the fallback fill.** Its structure depends on what stands + beside the box; a mirrored bystander is a bystander. +- **Reducing the switch as `CROP_CLEANUP`.** The banner would say "Crop & + cleanup" for a change that reruns the model; `SEGMENTATION` is the reason + a model change already uses and is the honest cost signal. diff --git a/docs/v1-scope.md b/docs/v1-scope.md index 9a1e461..9b92ff4 100644 --- a/docs/v1-scope.md +++ b/docs/v1-scope.md @@ -51,6 +51,7 @@ Nothing else gates V1. When that sentence is true end to end, V1 is done. | Output | Lossless animated WebP, plus the still-image single-frame case. | | Timeline | Filmstrip, playhead, IN/OUT range handles, exact time/frame readout. | | Crop | Visual rectangle with handles, plus numeric fields. | +| Exclusion regions | Several user-drawn rectangles over the source frame, forced to alpha 0 after segmentation so a burned-in overlay does not survive into the cutout. **Scope reopened 2026-09-12 (issue #80):** an optional global switch also blanks those regions before the model; effective boxes are part of the cut key, so changing them re-segments while post-only edits keep the stored cut set. | | Preview | `Preview Frame` through the exact render pipeline, with the stale-result contract. | | Jobs | Exclusive modal job dialog with truthful stage/progress and working cancellation. | | Accessibility | Full keyboard reachability, correct tab order, accessible names and values on **standard** widgets, no colour-only status. | @@ -77,6 +78,12 @@ Nothing else gates V1. When that sentence is true end to end, V1 is done. `Delete outdated` as the agreed handling for a rembg namespace move (issue #22). Bulk re-download returned in #29 with confirmation, cancellation, batch progress and per-model deletion after verification. +- **Exclusion regions carry across a source change, 2026-09-12:** the + rectangles are a parameter, not a per-source value, because the use case is + many clips of one stream sharing an overlay layout. They are clipped — not + slid — to each newly loaded source, so what the canvas paints and what the + renderer masks are the same rectangle by construction. Sliding, which is what + `clamp_crop` does for a crop, would move an exclusion onto wanted content. - **Cut workspace location, 2026-09-10:** durable cut sets and disposable scratch now live under MatteLoop's user cache at `/workspace/`, rather than inside diff --git a/resources/matteloop_de.qm b/resources/matteloop_de.qm index 9ea45c2..af374a7 100644 Binary files a/resources/matteloop_de.qm and b/resources/matteloop_de.qm differ diff --git a/resources/matteloop_de.ts b/resources/matteloop_de.ts index 8e9e951..84435b5 100644 --- a/resources/matteloop_de.ts +++ b/resources/matteloop_de.ts @@ -25,16 +25,86 @@ CropCanvas - + Original video frame Original-Videobild - + + Exclusion region mode: %1 region(s); no region selected + Ausschlussregion-Modus: %1 Region(en); keine Region ausgewählt + + + + + + ; blanked before the model + ; vor dem Modell ausgeblendet + + + + Exclusion region %1 of %2: x %3, y %4, width %5, height %6 source pixels; entirely outside crop + Ausschlussregion %1 von %2: x %3, y %4, Breite %5, Höhe %6 Quellpixel; vollständig außerhalb des Zuschnitts + + + + Exclusion region %1 of %2: x %3, y %4, width %5, height %6 source pixels; partly outside crop + Ausschlussregion %1 von %2: x %3, y %4, Breite %5, Höhe %6 Quellpixel; teilweise außerhalb des Zuschnitts + + + + Exclusion region %1 of %2: x %3, y %4, width %5, height %6 source pixels + Ausschlussregion %1 von %2: x %3, y %4, Breite %5, Höhe %6 Quellpixel + + + Crop bounds: x %s, y %s, width %s, height %s source pixels; source %s × %s Zuschnittgrenzen: x %s, y %s, Breite %s, Höhe %s Quellpixel; Quelle %s × %s + + ExclusionCanvas + + + Exclude this area + Diesen Bereich ausschließen + + + + Remove this region + Diese Region entfernen + + + + Remove all regions + Alle Regionen entfernen + + + + Edit exclusion regions + Ausschlussregionen bearbeiten + + + + ExclusionControls + + + %n region(s), blanked before the model + + %n Region, vor dem Modell ausgeblendet + %n Regionen, vor dem Modell ausgeblendet + + + + + + %n region(s) + + %n Region + %n Regionen + + + Inspector @@ -69,29 +139,29 @@ - - + + X X - - + + Y Y - - + + Width Breite - - + + Height Höhe @@ -118,31 +188,31 @@ - + Edge treatment Kantenbehandlung - + Output FPS Ausgabe-FPS - + Start Start - + End Ende - + Duration Dauer @@ -158,246 +228,303 @@ - + Horizontal stretch Horizontale Streckung + Exclusions + Ausschlussregionen + + + Directory Ordner - + Filename Dateiname - + Maximum size Maximale Größe - + Processing settings Verarbeitungseinstellungen - + Retry Rebuild Neuaufbau wiederholen - + Edited cut recovery Wiederherstellung bearbeiteter Schnitte - + Edited cut frames could not be validated. Retry the rebuild scan. Bearbeitete Schnittbilder konnten nicht validiert werden. Wiederholen Sie die Neuaufbauprüfung. - - + + Rebuild from edited cuts Aus bearbeiteten Schnitten neu aufbauen - - + + Manage Models… Modelle verwalten… - - + + Manage Workspaces… Arbeitsbereiche verwalten… - + Reset to defaults Auf Standardwerte zurücksetzen - + Reset inspector parameters Inspector-Parameter zurücksetzen - + Compute acceleration, transform, output directory, and output filename are not affected. Beschleunigung, Transformation, Ausgabeordner und Dateiname bleiben unverändert. - + Segmentation model Segmentierungsmodell - + cached locally lokal zwischengespeichert - + not cached yet noch nicht zwischengespeichert - + Model status Modellstatus - + Model download Modelldownload - + Standard Standard - + Decontaminate colors Farben bereinigen - + fps FPS - + High output FPS may increase render cost Hohe Ausgabe-FPS können den Renderaufwand erhöhen - + Output FPS cost warning Warnung zum Aufwand der Ausgabe-FPS - + Global trim Globaler Randbeschnitt - + Global alpha trim Globaler Alpha-Beschnitt - + Padding pixels Randpixel - + Output directory Ausgabeordner - + Choose… Auswählen… - + Choose output directory Ausgabeordner auswählen - + + No region selected + Keine Region ausgewählt + + + + Selected region: x %1, y %2, width %3, height %4 source pixels; entirely outside crop + Ausgewählte Region: x %1, y %2, Breite %3, Höhe %4 Quellpixel; vollständig außerhalb des Zuschnitts + + + + Selected region: x %1, y %2, width %3, height %4 source pixels; partly outside crop + Ausgewählte Region: x %1, y %2, Breite %3, Höhe %4 Quellpixel; teilweise außerhalb des Zuschnitts + + + + Selected region: x %1, y %2, width %3, height %4 source pixels + Ausgewählte Region: x %1, y %2, Breite %3, Höhe %4 Quellpixel + + + + Edit + Bearbeiten + + + + + Edit exclusion regions + Ausschlussregionen bearbeiten + + + + Clear Zurücksetzen - + + Clear exclusion regions + Ausschlussregionen löschen + + + + Blank before the model + Vor dem Modell ausblenden + + + + Re-segments the clip: the stored cut set is not reused + Der Clip wird neu segmentiert: Der gespeicherte Schnitt wird nicht wiederverwendet + + + + Selected exclusion region + Ausgewählte Ausschlussregion + + + Clear output directory Ausgabeordner zurücksetzen - + Output filename Ausgabedateiname - - + + Use one non-empty filename ending in .webp; path separators are not allowed. Verwenden Sie einen einzelnen, nicht leeren Dateinamen mit der Endung .webp; Pfadtrenner sind nicht erlaubt. - + Maximum file size Maximale Dateigröße - + Unlimited Unbegrenzt - + MiB MiB - + Unlimited means no maximum file size. Unbegrenzt bedeutet: Es gibt keine maximale Dateigröße. - + %s — %s download required %s — %s Download erforderlich - + Filename must be a single non-empty .webp filename. Der Dateiname muss ein einzelner, nicht leerer .webp-Dateiname sein. - + s s - + Crop Zuschnitt - + Crop overlay Zuschnitt-Overlay - + Reset Crop Zuschnitt zurücksetzen - + Reset crop Zuschnitt zurücksetzen - + Crop %s Zuschnitt %s - + Model status: %s Modellstatus: %s - + Open a video to manage its workspaces. Öffnen Sie ein Video, um dessen Arbeitsbereiche zu verwalten. @@ -448,7 +575,7 @@ Ordner öffnen - + Segmentation is unavailable because ONNX Runtime could not enumerate any providers. Repair the installation with: %1 Segmentierung ist nicht verfügbar, weil ONNX Runtime keine Provider ermitteln konnte. Reparieren Sie die Installation mit: %1 @@ -1883,39 +2010,39 @@ Freed %s. ResultPlayerCanvas - + Result Ergebnis - - + + Play Wiedergeben - - + + Play the result loop Ergebniswiedergabe abspielen - + Pause Pause - + Pause the result loop Ergebniswiedergabe pausieren - + Previewing the first %s of %s frames Die ersten %s von %s Bildern werden in der Vorschau angezeigt - + Cut frames could not be read Schnittbilder konnten nicht gelesen werden @@ -2054,7 +2181,7 @@ Freed %s. Videodateien (*.mp4 *.mov *.webm *.mkv) - + Choose output directory Ausgabeordner auswählen @@ -2510,12 +2637,12 @@ Freed %s. TransformStage - + Discard unsaved transform changes? Ungespeicherte Transformationsänderungen verwerfen? - + The current cut has unsaved transform changes. Discard them and continue? Das aktuelle Schnittset enthält ungespeicherte Transformationsänderungen. Sollen sie verworfen und fortgefahren werden? @@ -2881,7 +3008,7 @@ in %1; %2; %3; %4 bei %5; %6; erstellt %7; Zweck: %8; Lizenz: %9; %10 - + unknown time unbekannte Zeit diff --git a/resources/matteloop_en.qm b/resources/matteloop_en.qm index 38ce644..3935733 100644 Binary files a/resources/matteloop_en.qm and b/resources/matteloop_en.qm differ diff --git a/resources/matteloop_en.ts b/resources/matteloop_en.ts index 53126ae..3465b68 100644 --- a/resources/matteloop_en.ts +++ b/resources/matteloop_en.ts @@ -25,16 +25,86 @@ CropCanvas - + Original video frame Original video frame - + + Exclusion region mode: %1 region(s); no region selected + Exclusion region mode: %1 region(s); no region selected + + + + + + ; blanked before the model + ; blanked before the model + + + + Exclusion region %1 of %2: x %3, y %4, width %5, height %6 source pixels; entirely outside crop + Exclusion region %1 of %2: x %3, y %4, width %5, height %6 source pixels; entirely outside crop + + + + Exclusion region %1 of %2: x %3, y %4, width %5, height %6 source pixels; partly outside crop + Exclusion region %1 of %2: x %3, y %4, width %5, height %6 source pixels; partly outside crop + + + + Exclusion region %1 of %2: x %3, y %4, width %5, height %6 source pixels + Exclusion region %1 of %2: x %3, y %4, width %5, height %6 source pixels + + + Crop bounds: x %s, y %s, width %s, height %s source pixels; source %s × %s Crop bounds: x %s, y %s, width %s, height %s source pixels; source %s × %s + + ExclusionCanvas + + + Exclude this area + Exclude this area + + + + Remove this region + Remove this region + + + + Remove all regions + Remove all regions + + + + Edit exclusion regions + Edit exclusion regions + + + + ExclusionControls + + + %n region(s), blanked before the model + + %n region(s), blanked before the model + %n region(s), blanked before the model + + + + + + %n region(s) + + %n region + %n regions + + + Inspector @@ -69,29 +139,29 @@ - - + + X X - - + + Y Y - - + + Width Width - - + + Height Height @@ -118,31 +188,31 @@ - + Edge treatment Edge treatment - + Output FPS Output FPS - + Start Start - + End End - + Duration Duration @@ -158,246 +228,303 @@ - + Horizontal stretch Horizontal stretch + Exclusions + Exclusions + + + Directory Directory - + Filename Filename - + Maximum size Maximum size - + Processing settings Processing settings - + Retry Rebuild Retry Rebuild - + Edited cut recovery Edited cut recovery - + Edited cut frames could not be validated. Retry the rebuild scan. Edited cut frames could not be validated. Retry the rebuild scan. - - + + Rebuild from edited cuts Rebuild from edited cuts - - + + Manage Models… Manage Models… - - + + Manage Workspaces… Manage Workspaces… - + Reset to defaults Reset to defaults - + Reset inspector parameters Reset inspector parameters - + Compute acceleration, transform, output directory, and output filename are not affected. Compute acceleration, transform, output directory, and output filename are not affected. - + Segmentation model Segmentation model - + cached locally cached locally - + not cached yet not cached yet - + Model status Model status - + Model download Model download - + Standard Standard - + Decontaminate colors Decontaminate colors - + fps fps - + High output FPS may increase render cost High output FPS may increase render cost - + Output FPS cost warning Output FPS cost warning - + Global trim Global trim - + Global alpha trim Global alpha trim - + Padding pixels Padding pixels - + Output directory Output directory - + Choose… Choose… - + Choose output directory Choose output directory - + + No region selected + No region selected + + + + Selected region: x %1, y %2, width %3, height %4 source pixels; entirely outside crop + Selected region: x %1, y %2, width %3, height %4 source pixels; entirely outside crop + + + + Selected region: x %1, y %2, width %3, height %4 source pixels; partly outside crop + Selected region: x %1, y %2, width %3, height %4 source pixels; partly outside crop + + + + Selected region: x %1, y %2, width %3, height %4 source pixels + Selected region: x %1, y %2, width %3, height %4 source pixels + + + + Edit + Edit + + + + + Edit exclusion regions + Edit exclusion regions + + + + Clear Clear - + + Clear exclusion regions + Clear exclusion regions + + + + Blank before the model + Blank before the model + + + + Re-segments the clip: the stored cut set is not reused + Re-segments the clip: the stored cut set is not reused + + + + Selected exclusion region + Selected exclusion region + + + Clear output directory Clear output directory - + Output filename Output filename - - + + Use one non-empty filename ending in .webp; path separators are not allowed. Use one non-empty filename ending in .webp; path separators are not allowed. - + Maximum file size Maximum file size - + Unlimited Unlimited - + MiB MiB - + Unlimited means no maximum file size. Unlimited means no maximum file size. - + %s — %s download required %s — %s download required - + Filename must be a single non-empty .webp filename. Filename must be a single non-empty .webp filename. - + s s - + Crop Crop - + Crop overlay Crop overlay - + Reset Crop Reset Crop - + Reset crop Reset crop - + Crop %s Crop %s - + Model status: %s Model status: %s - + Open a video to manage its workspaces. Open a video to manage its workspaces. @@ -448,7 +575,7 @@ Open folder - + Segmentation is unavailable because ONNX Runtime could not enumerate any providers. Repair the installation with: %1 Segmentation is unavailable because ONNX Runtime could not enumerate any providers. Repair the installation with: %1 @@ -1883,39 +2010,39 @@ Freed %s. ResultPlayerCanvas - + Result Result - - + + Play Play - - + + Play the result loop Play the result loop - + Pause Pause - + Pause the result loop Pause the result loop - + Previewing the first %s of %s frames Previewing the first %s of %s frames - + Cut frames could not be read Cut frames could not be read @@ -2054,7 +2181,7 @@ Freed %s. Video files (*.mp4 *.mov *.webm *.mkv) - + Choose output directory Choose output directory @@ -2510,12 +2637,12 @@ Freed %s. TransformStage - + Discard unsaved transform changes? Discard unsaved transform changes? - + The current cut has unsaved transform changes. Discard them and continue? The current cut has unsaved transform changes. Discard them and continue? @@ -2881,7 +3008,7 @@ into %1; %2; %3; %4 at %5; %6; created %7; purpose: %8; licence: %9; %10 - + unknown time unknown time diff --git a/scripts/guardrails-baseline.json b/scripts/guardrails-baseline.json index ef42f3d..f23e809 100644 --- a/scripts/guardrails-baseline.json +++ b/scripts/guardrails-baseline.json @@ -4,7 +4,7 @@ "long_functions": 1 }, "src/matteloop/core/state.py": { - "lines": 882, + "lines": 883, "long_functions": 1 }, "src/matteloop/core/webp.py": { @@ -24,7 +24,7 @@ "long_functions": 1 }, "src/matteloop/jobs/render.py": { - "lines": 2886, + "lines": 2889, "long_functions": 11 }, "src/matteloop/jobs/segmentation_host.py": { @@ -88,7 +88,7 @@ "long_functions": 1 }, "src/matteloop/ui/inspector.py": { - "lines": 897, + "lines": 904, "long_functions": 0 }, "src/matteloop/ui/timeline.py": { diff --git a/scripts/screenshots.py b/scripts/screenshots.py index 24ac54c..260cac2 100644 --- a/scripts/screenshots.py +++ b/scripts/screenshots.py @@ -29,6 +29,7 @@ from PySide6.QtWidgets import QApplication, QStyleFactory, QWidget from matteloop.core.parameters import ParameterState +from matteloop.core.specs import CropSpec from matteloop.core.state import ( AppState, PreviewRequested, @@ -123,6 +124,8 @@ def _state(metadata: object, source_frame: QImage, result_frame: QImage) -> AppS parameters=ParameterState( model_id=_MODEL_ID, output_directory=_OUTPUT_DIRECTORY, + exclusions=(CropSpec(560, 32, 150, 88),), + exclusions_before_model=True, ) ), SourceLoadRequested(_SOURCE_ID, _LOAD_REQUEST_ID), diff --git a/src/matteloop/core/crop.py b/src/matteloop/core/crop.py index 8b4d671..b47b905 100644 --- a/src/matteloop/core/crop.py +++ b/src/matteloop/core/crop.py @@ -3,6 +3,7 @@ from __future__ import annotations import math +from dataclasses import replace from fractions import Fraction from matteloop.core.geometry import ( @@ -255,15 +256,23 @@ def _orientation_transform( source_width: int, source_height: int, rotation: int, pixel_aspect: float ) -> MediaTransform: _validate_dimensions(source_width, source_height) - return MediaTransform( + transform = MediaTransform( source_size=SizeF(source_width, source_height), - viewport=SizeF( - source_height if rotation in {90, 270} else source_width, - source_width * pixel_aspect if rotation in {90, 270} else source_height, - ), + viewport=SizeF(source_width, source_height), rotation=rotation, pixel_aspect=pixel_aspect, ) + display_width = max( + 1, + _round_fraction(Fraction(source_width) * Fraction(str(pixel_aspect))), + ) + return replace( + transform, + viewport=SizeF( + source_height if rotation in {90, 270} else display_width, + display_width if rotation in {90, 270} else source_height, + ), + ) def _orientation_transform_from_media(transform: MediaTransform) -> MediaTransform: @@ -279,6 +288,11 @@ def _rounded_delta(value: float) -> int: return math.floor(value + 0.5) if value >= 0 else math.ceil(value - 0.5) +def _round_fraction(value: Fraction) -> int: + quotient, remainder = divmod(value.numerator, value.denominator) + return quotient + int(remainder * 2 >= value.denominator) + + def _validate_dimensions(source_width: int, source_height: int) -> None: if ( not isinstance(source_width, int) diff --git a/src/matteloop/core/exclusion.py b/src/matteloop/core/exclusion.py new file mode 100644 index 0000000..749ddc5 --- /dev/null +++ b/src/matteloop/core/exclusion.py @@ -0,0 +1,118 @@ +"""Map and apply post-segmentation exclusion regions. + +All cut pixels enter framing through three producers: + +* ``_produce_cut_frame`` (sites 1, 2) +* ``WorkspacePort.read_cut`` (sites 3, 4) +* ``FrameReader.read`` (sites 5, 6) + +The six consumers are: + +1. preview (``render.py:913-960``): ``exclude_alpha(cut, boxes)`` once, + right after ``_produce_cut_frame``; bounds, both ``apply_framing`` calls, + and both ``_immutable_rgba`` images see it. +2. render loop (``render.py:1108-1112``): stage first, then mask in place. +3. ``_rebuild_union`` (``render.py:1338``): ``exclude_alpha(image, boxes)`` + after ``read_cut``. +4. encoder (``render.py:1386``): ``stage_encoder_frames`` passes boxes to + ``_stage_frame``, which masks before ``apply_framing``. +5. UI facts (``ui/transform_stage.py:710-720``): ``_iter_stored_frames`` + masks before yielding. +6. result player (``ui/result_player.py:480-490``): ``FrameLoadWorker`` and + ``_load_one_frame`` mask before ``apply_framing``. + +``union_alpha_bounds_or_none`` keeps a genuinely empty alpha union distinct +from malformed or mismatched framing inputs. +""" + +from __future__ import annotations + +from collections.abc import Iterable +from decimal import Decimal + +from PIL import Image + +from matteloop.core.errors import ErrorCode, ValidationError +from matteloop.core.geometry import PixelBounds, alpha_bounds +from matteloop.core.specs import CropSpec + +MODEL_FILL_ID = "imagenet-mean" +MODEL_FILL_RGB = (124, 116, 104) + + +def cut_exclusions( + exclusions: tuple[CropSpec, ...], crop: CropSpec +) -> tuple[PixelBounds, ...]: + """Map oriented-source regions into the crop-sized cut, clipped and canonical.""" + boxes: set[tuple[int, int, int, int]] = set() + for region in exclusions: + left = max(region.x, crop.x) - crop.x + top = max(region.y, crop.y) - crop.y + right = min(region.x + region.width, crop.x + crop.width) - crop.x + bottom = min(region.y + region.height, crop.y + crop.height) - crop.y + if right > left and bottom > top: + boxes.add((left, top, right, bottom)) + return tuple(PixelBounds(*box) for box in sorted(boxes)) + + +def blank_model_input( + image: Image.Image, boxes: tuple[PixelBounds, ...] +) -> None: + """Fill model-excluded RGB pixels while preserving their alpha channel.""" + if not boxes: + return + alpha = image.getchannel("A") + try: + for box in boxes: + image.paste( + MODEL_FILL_RGB, + (box.left, box.top, box.right, box.bottom), + ) + image.putalpha(alpha) + finally: + alpha.close() + + +def exclude_alpha(image: Image.Image, boxes: tuple[PixelBounds, ...]) -> None: + """Zero RGBA inside each box, in place.""" + for box in boxes: + image.paste((0, 0, 0, 0), (box.left, box.top, box.right, box.bottom)) + + +def union_alpha_bounds_or_none( + frames: Iterable[Image.Image], threshold_percent: Decimal | float | int +) -> PixelBounds | None: + """Return a valid alpha union, or ``None`` only when it is empty.""" + try: + iterator = iter(frames) + except TypeError: + raise ValidationError( + ErrorCode.INVALID_FRAMING, + "framing", + "framing requires an iterable of RGBA frames", + ) from None + + union: PixelBounds | None = None + expected_size: tuple[int, int] | None = None + for image in iterator: + bounds = alpha_bounds(image, threshold_percent) + if expected_size is None: + expected_size = image.size + elif image.size != expected_size: + raise ValidationError( + ErrorCode.INVALID_FRAMING, + "framing", + "all frames must have identical source canvas dimensions", + ) + if bounds is None: + continue + if union is None: + union = bounds + else: + union = PixelBounds( + min(union.left, bounds.left), + min(union.top, bounds.top), + max(union.right, bounds.right), + max(union.bottom, bounds.bottom), + ) + return union diff --git a/src/matteloop/core/fingerprints.py b/src/matteloop/core/fingerprints.py index f1cb62d..33f6c78 100644 --- a/src/matteloop/core/fingerprints.py +++ b/src/matteloop/core/fingerprints.py @@ -14,6 +14,7 @@ from typing import BinaryIO, Protocol, cast from matteloop.core.errors import AppError, ErrorCode, ValidationError +from matteloop.core.exclusion import MODEL_FILL_ID, cut_exclusions from matteloop.core.specs import ( CropSpec, FramingSpec, @@ -154,8 +155,9 @@ def preview_fingerprint( "model_id": request.segmentation.model_id, "model_weight_sha256": model_weight_sha256, **_edge_settings(request), + **_model_exclusions(request), }, - "framing": _framing(request.framing), + "framing": _framing(request), "pipeline_schema_version": pipeline_schema_version, "orientation_color_version": orientation_color_version, "rembg_version": rembg_version, @@ -217,6 +219,7 @@ def cut_cache_key_inputs( "pipeline_schema_version": pipeline_schema_version, "orientation_color_version": orientation_color_version, "edge_settings": _edge_settings(request), + **_model_exclusions(request), } ) @@ -240,7 +243,7 @@ def render_fingerprint(request: RenderRequest, *, cut_key: str) -> str: { **_schema("render"), "cut_key": cut_key, - "framing": _framing(request.framing), + "framing": _framing(request), "max_bytes": request.output.max_bytes, } ) @@ -249,14 +252,55 @@ def render_fingerprint(request: RenderRequest, *, cut_key: str) -> str: def union_fingerprint(request: RenderRequest, *, cut_key: str) -> str: """Identify the cut bytes and threshold consumed by range-union analysis.""" request = _validated_request(request) + return cut_union_fingerprint(request.framing, request.crop, cut_key=cut_key) + + +def cut_union_fingerprint( + framing: FramingSpec, crop: CropSpec, *, cut_key: str +) -> str: + """Identify cut-union inputs, including effective exclusions when present.""" cut_key = _validated_sha256(cut_key, "cut_key") - return _canonical_hash( - { - **_schema("cut-union"), - "cut_key": cut_key, - "alpha_threshold": _canonical_decimal(request.framing.alpha_threshold), + payload: dict[str, object] = { + **_schema("cut-union"), + "cut_key": cut_key, + "alpha_threshold": _canonical_decimal(framing.alpha_threshold), + } + boxes = cut_exclusions(framing.exclusions, crop) + if boxes: + payload["exclusions"] = [ + [box.left, box.top, box.right, box.bottom] for box in boxes + ] + return _canonical_hash(payload) + + +def _framing(request: RenderRequest) -> dict[str, object]: + framing = request.framing + payload: dict[str, object] = { + "trim": framing.trim, + "alpha_threshold": _canonical_decimal(framing.alpha_threshold), + "padding": framing.padding, + "stretch_x": _canonical_decimal(framing.stretch_x), + } + boxes = cut_exclusions(framing.exclusions, request.crop) + if boxes: + payload["exclusions"] = [ + [box.left, box.top, box.right, box.bottom] for box in boxes + ] + return payload + + +def _model_exclusions(request: RenderRequest) -> dict[str, object]: + boxes = cut_exclusions(request.segmentation.exclusions, request.crop) + if not boxes: + return {} + return { + "model_exclusions": { + "fill": MODEL_FILL_ID, + "boxes": tuple( + (box.left, box.top, box.right, box.bottom) for box in boxes + ), } - ) + } def _update_digest( @@ -339,15 +383,6 @@ def _crop(crop: CropSpec) -> dict[str, int]: return {"x": crop.x, "y": crop.y, "width": crop.width, "height": crop.height} -def _framing(framing: FramingSpec) -> dict[str, object]: - return { - "trim": framing.trim, - "alpha_threshold": _canonical_decimal(framing.alpha_threshold), - "padding": framing.padding, - "stretch_x": _canonical_decimal(framing.stretch_x), - } - - def _edge_settings(request: RenderRequest) -> dict[str, object]: matting = request.segmentation.alpha_matting return { diff --git a/src/matteloop/core/parameters.py b/src/matteloop/core/parameters.py index 67dc924..dd225c5 100644 --- a/src/matteloop/core/parameters.py +++ b/src/matteloop/core/parameters.py @@ -10,11 +10,13 @@ from typing import TYPE_CHECKING from matteloop.core.errors import ValidationError +from matteloop.core.exclusion import cut_exclusions from matteloop.core.execution_providers import ( CPU_EXECUTION_PROVIDER, is_allowed_provider, ) from matteloop.core.specs import ( + CropSpec, EdgeMode, FramingSpec, OutputSpec, @@ -56,6 +58,8 @@ class ParameterState: alpha_threshold: Decimal = Decimal("2.0") padding: int = 0 stretch_x: Decimal = Decimal("1.0") + exclusions: tuple[CropSpec, ...] = () + exclusions_before_model: bool = False output_directory: Path | None = None output_filename: str | None = None max_mib: Decimal = Decimal("0") @@ -74,6 +78,7 @@ def __post_init__(self) -> None: self.alpha_threshold, self.padding, self.stretch_x, + self.exclusions, ) if self.output_directory is not None and ( not isinstance(self.output_directory, Path) @@ -129,6 +134,16 @@ class StretchChanged: value: Decimal +@dataclass(frozen=True, slots=True) +class ExclusionsChanged: + exclusions: tuple[CropSpec, ...] + + +@dataclass(frozen=True, slots=True) +class ExclusionsBeforeModelChanged: + enabled: bool + + @dataclass(frozen=True, slots=True) class OutputDirectoryChanged: directory: Path | None @@ -163,6 +178,8 @@ class TransformChanged: | AlphaThresholdChanged | PaddingChanged | StretchChanged + | ExclusionsChanged + | ExclusionsBeforeModelChanged | OutputDirectoryChanged | OutputFilenameChanged | OutputMaxSizeChanged @@ -198,6 +215,10 @@ def reduce_parameters(state: AppState, event: ParameterEvent) -> AppState: return _reduce_cleanup_value(state, event) if isinstance(event, StretchChanged): return _reduce_cleanup_value(state, event) + if isinstance(event, ExclusionsChanged): + return _reduce_exclusions(state, event) + if isinstance(event, ExclusionsBeforeModelChanged): + return _reduce_exclusions_before_model(state, event) if isinstance(event, OutputDirectoryChanged): return _reduce_output_directory(state, event) if isinstance(event, OutputFilenameChanged): @@ -222,6 +243,8 @@ def _reduce_parameters_reset(state: AppState) -> AppState: alpha_threshold=defaults.alpha_threshold, padding=defaults.padding, stretch_x=defaults.stretch_x, + exclusions=defaults.exclusions, + exclusions_before_model=defaults.exclusions_before_model, max_mib=defaults.max_mib, ) timeline = state.timeline @@ -242,6 +265,8 @@ def _reduce_parameters_reset(state: AppState) -> AppState: if ( parameters.model_id != state.parameters.model_id or parameters.edge_mode != state.parameters.edge_mode + or parameters.exclusions_before_model + != state.parameters.exclusions_before_model ): reason = PreviewInvalidationReason.SEGMENTATION elif parameters.fps != state.parameters.fps or ( @@ -257,6 +282,7 @@ def _reduce_parameters_reset(state: AppState) -> AppState: (parameters.alpha_threshold, state.parameters.alpha_threshold), (parameters.padding, state.parameters.padding), (parameters.stretch_x, state.parameters.stretch_x), + (parameters.exclusions, state.parameters.exclusions), ) ): reason = PreviewInvalidationReason.CROP_CLEANUP @@ -366,6 +392,52 @@ def _reduce_cleanup_value( return _invalidate(state, updated, PreviewInvalidationReason.CROP_CLEANUP) +def _reduce_exclusions(state: AppState, event: ExclusionsChanged) -> AppState: + from matteloop.core.tokens import PreviewInvalidationReason + + if not isinstance(event.exclusions, tuple) or any( + not isinstance(exclusion, CropSpec) for exclusion in event.exclusions + ): + return state + updated = replace(state.parameters, exclusions=event.exclusions) + if updated == state.parameters: + return state + crop = state.crop + if crop is None or cut_exclusions(event.exclusions, crop) == cut_exclusions( + state.parameters.exclusions, crop + ): + return replace(state, parameters=updated) + return _invalidate( + state, + updated, + ( + PreviewInvalidationReason.SEGMENTATION + if state.parameters.exclusions_before_model + else PreviewInvalidationReason.CROP_CLEANUP + ), + ) + + +def _reduce_exclusions_before_model( + state: AppState, event: ExclusionsBeforeModelChanged +) -> AppState: + from matteloop.core.tokens import PreviewInvalidationReason + + if ( + type(event.enabled) is not bool + or event.enabled == state.parameters.exclusions_before_model + ): + return state + updated = replace( + state.parameters, exclusions_before_model=event.enabled + ) + if state.crop is None or not cut_exclusions( + state.parameters.exclusions, state.crop + ): + return replace(state, parameters=updated) + return _invalidate(state, updated, PreviewInvalidationReason.SEGMENTATION) + + def _reduce_output_directory( state: AppState, event: OutputDirectoryChanged ) -> AppState: @@ -451,6 +523,10 @@ def parameters_from_values(values: Mapping[str, object]) -> ParameterState: stretch_x=_decimal_value( values.get("stretch_x"), defaults.stretch_x, lambda value: value > 0 ), + exclusions=_exclusions_value(values.get("exclusions"), defaults.exclusions), + exclusions_before_model=_bool_value( + values.get("exclusions_before_model"), defaults.exclusions_before_model + ), max_mib=_decimal_value( values.get("max_mib"), defaults.max_mib, lambda value: value >= 0 ), @@ -540,6 +616,54 @@ def _bool_value(value: object, default: bool) -> bool: return default +def _exclusions_value( + value: object, default: tuple[CropSpec, ...] +) -> tuple[CropSpec, ...]: + if value is None: + return default + if not isinstance(value, str): + return () + if not value: + return () + try: + exclusions = [] + for entry in value.split(";"): + fields = entry.split(",") + if len(fields) != 4: + return () + exclusions.append(CropSpec(*(int(field) for field in fields))) + return tuple(exclusions) + except (TypeError, ValueError, ValidationError): + return () + + +def clip_exclusions(parameters: ParameterState, source: object) -> ParameterState: + """Intersect carried regions with the newly loaded source dimensions.""" + width = getattr(source, "width", None) + height = getattr(source, "height", None) + if type(width) is not int or type(height) is not int or width < 1 or height < 1: + return parameters + clipped: list[CropSpec] = [] + for exclusion in parameters.exclusions: + right = min(exclusion.x + exclusion.width, width) + bottom = min(exclusion.y + exclusion.height, height) + if exclusion.x < width and exclusion.y < height: + clipped.append( + CropSpec( + exclusion.x, + exclusion.y, + right - exclusion.x, + bottom - exclusion.y, + ) + ) + result = tuple(clipped) + return ( + parameters + if result == parameters.exclusions + else replace(parameters, exclusions=result) + ) + + def _directory_value(value: object) -> Path | None: if not isinstance(value, (str, Path)) or not str(value): return None diff --git a/src/matteloop/core/specs.py b/src/matteloop/core/specs.py index 13746a4..616d8ee 100644 --- a/src/matteloop/core/specs.py +++ b/src/matteloop/core/specs.py @@ -209,6 +209,7 @@ class SegmentationSpec: edge_mode: EdgeMode = EdgeMode.STANDARD alpha_matting: AlphaMattingSpec = field(default_factory=AlphaMattingSpec) execution_provider: str = CPU_EXECUTION_PROVIDER + exclusions: tuple[CropSpec, ...] = () def __post_init__(self) -> None: self.validate() @@ -238,6 +239,14 @@ def validate(self) -> None: "segmentation", "alpha_matting must be an AlphaMattingSpec", ) + if not isinstance(self.exclusions, tuple) or any( + not isinstance(exclusion, CropSpec) for exclusion in self.exclusions + ): + raise ValidationError( + ErrorCode.INVALID_SEGMENTATION, + "segmentation", + "exclusions must be a tuple of CropSpec values", + ) @dataclass(frozen=True) @@ -248,6 +257,7 @@ class FramingSpec: alpha_threshold: Decimal = Decimal("2.0") padding: int = 0 stretch_x: Decimal = Decimal("1.0") + exclusions: tuple[CropSpec, ...] = () def __post_init__(self) -> None: self.validate() @@ -287,6 +297,14 @@ def validate(self) -> None: "framing", "horizontal stretch must be a positive Decimal", ) + if not isinstance(self.exclusions, tuple) or any( + not isinstance(exclusion, CropSpec) for exclusion in self.exclusions + ): + raise ValidationError( + ErrorCode.INVALID_FRAMING, + "framing", + "exclusions must be a tuple of CropSpec values", + ) def validate_final_dimensions(self, width: int, height: int) -> tuple[int, int]: """Validate fixed WebP dimensions before allocation or output replacement.""" diff --git a/src/matteloop/core/state.py b/src/matteloop/core/state.py index 54cf23f..8925b8d 100644 --- a/src/matteloop/core/state.py +++ b/src/matteloop/core/state.py @@ -403,6 +403,7 @@ def reduce(state: AppState, event: Event) -> AppState: source_frame=event.frame, timeline=timeline_from_metadata(event.value, state.parameters.fps), crop=_crop_state.default_crop_for_source(event.value), source_error=None, + parameters=_parameters.clip_exclusions(state.parameters, event.value), focus_target=FocusTarget.PREVIEW_ACTION, ) if isinstance(event, _crop_state.CropEvent): diff --git a/src/matteloop/jobs/render.py b/src/matteloop/jobs/render.py index 1dda189..c00bc77 100644 --- a/src/matteloop/jobs/render.py +++ b/src/matteloop/jobs/render.py @@ -7,7 +7,6 @@ | | v v durable promote framing -> encode - rebuild: durable cuts -----------------> private snapshot -> framing -> encode ``_produce_cut_frame`` is the only decode/crop/segment/edge-cleanup path. The @@ -36,6 +35,12 @@ from PIL import Image from matteloop.core.errors import AppError, ErrorCode, ValidationError +from matteloop.core.exclusion import ( + blank_model_input, + cut_exclusions, + exclude_alpha, + union_alpha_bounds_or_none, +) from matteloop.core.fingerprints import ( PIPELINE_SCHEMA_VERSION, REMBG_VERSION, @@ -79,7 +84,12 @@ from matteloop.jobs.models.cache_fs import BoundDirectoryCloseError, UnsafeCacheError from matteloop.jobs.protocol import PROTOCOL_VERSION, SegmentOptions, SegmentRequest from matteloop.jobs.source import DecodedFrame, SourceInfo, decode_frame, probe_source -from matteloop.jobs.transform_stage import framing_plan, stage_encoder_frames +from matteloop.jobs.transform_stage import ( + _iter_rebuild_frames, + cached_union, + framing_plan, + stage_encoder_frames, +) from matteloop.jobs.transform_store import store_transform from matteloop.jobs.workspace import ( AdvisoryFileLock, @@ -921,6 +931,8 @@ def preview( tracker, ) try: + cut_boxes = cut_exclusions(request.framing.exclusions, request.crop) + exclude_alpha(cut, cut_boxes) local_bounds = alpha_bounds(cut, request.framing.alpha_threshold) applied_bounds: PixelBounds | None = None exact = False @@ -988,14 +1000,7 @@ def _matching_union( ) manifest = self._workspace.validate(workspace) metadata = manifest.union_metadata - if ( - metadata is None - or metadata.fingerprint != union_fingerprint(request, cut_key=cache_key) - or metadata.alpha_threshold - != _decimal_text(request.framing.alpha_threshold) - ): - return None - return PixelBounds(*metadata.bounds) + return cached_union(metadata, request.framing, request.crop, cache_key) except AppError as error: if error.code in { ErrorCode.CUT_MANIFEST_INVALID, @@ -1078,6 +1083,7 @@ def render(self, request: RenderRequest, context: JobContext) -> RenderArtifact: ) cache_key = cut_cache_key_from_inputs(inputs) tracker = RgbaOwnershipTracker((request.crop.width, request.crop.height)) + exclusions = cut_exclusions(request.framing.exclusions, request.crop) self._advisory_disk_check(request, len(timestamps), notes) context.checkpoint("cut-staging") staged = self._workspace.create_staging( @@ -1106,9 +1112,10 @@ def render(self, request: RenderRequest, context: JobContext) -> RenderArtifact: tracker, ) try: + frame_records.append(self._workspace.stage(staged, index, cut)) + exclude_alpha(cut, exclusions) bounds = alpha_bounds(cut, request.framing.alpha_threshold) union = _union_bounds(union, bounds) - frame_records.append(self._workspace.stage(staged, index, cut)) actual_pts.append(actual) finally: cut.close() @@ -1323,32 +1330,20 @@ def _rebuild_union( notes: list[str], tracker: RgbaOwnershipTracker, ) -> PixelBounds | None: - metadata = manifest.union_metadata - expected_fingerprint = union_fingerprint(request, cut_key=manifest.cache_key) - if ( - metadata is not None - and metadata.fingerprint == expected_fingerprint - and metadata.alpha_threshold - == _decimal_text(request.framing.alpha_threshold) - ): - return PixelBounds(*metadata.bounds) - union: PixelBounds | None = None - for index in range(manifest.frame_count): - context.checkpoint("rebuild-union") - image = self._workspace.read_cut(private, index, tracker) - try: - union = _union_bounds( - union, - alpha_bounds(image, request.framing.alpha_threshold), - ) - finally: - image.close() - del image + cached = cached_union( + manifest.union_metadata, request.framing, request.crop, manifest.cache_key + ) + if cached is not None: + return cached + boxes = cut_exclusions(request.framing.exclusions, request.crop) + store = self._workspace + frames = _iter_rebuild_frames(store, private, manifest, tracker, context, boxes) + union = union_alpha_bounds_or_none(frames, request.framing.alpha_threshold) if union is not None: new_metadata = CutUnionMetadata( (union.left, union.top, union.right, union.bottom), _decimal_text(request.framing.alpha_threshold), - expected_fingerprint, + union_fingerprint(request, cut_key=manifest.cache_key), ) try: won = self._workspace.compare_and_set_union( @@ -1380,9 +1375,13 @@ def _encode_snapshot( overall: tuple[int, int], rebuilt: bool, ) -> RenderArtifact: + if request.framing.trim and union is None: + notes.append("trim skipped: no pixel above the alpha threshold " + + "survives the exclusion regions") plan = framing_plan((manifest.width, manifest.height), union, request.framing) tracker.include_size(plan.output_size) scratch = private.path.parent + exclusions = cut_exclusions(request.framing.exclusions, request.crop) framed_paths, delays = stage_encoder_frames( partial(self._workspace.read_cut, private), manifest.frame_count, @@ -1393,6 +1392,7 @@ def _encode_snapshot( tracker, context, overall=overall, + exclusions=exclusions, ) candidate = self._output_publisher.candidate_path( request.output.path, context.job_id, scratch @@ -1561,6 +1561,8 @@ def _produce_cut_frame( decoded_image.close() del decoded, decoded_image try: + model_boxes = cut_exclusions(request.segmentation.exclusions, request.crop) + blank_model_input(cropped, model_boxes) input_frame = np.ascontiguousarray(np.asarray(cropped, dtype=np.uint8)) tracker.register(input_frame) options = _segment_options(request) @@ -1604,6 +1606,7 @@ def _produce_cut_frame( segmented.tobytes(order="C"), ) tracker.register(result) + exclude_alpha(result, model_boxes) del segmented return result, actual_pts diff --git a/src/matteloop/jobs/transform_stage.py b/src/matteloop/jobs/transform_stage.py index 417a52d..f9bc40c 100644 --- a/src/matteloop/jobs/transform_stage.py +++ b/src/matteloop/jobs/transform_stage.py @@ -17,32 +17,61 @@ import dataclasses import os import tempfile -from collections.abc import Callable +from collections.abc import Callable, Iterator +from contextlib import closing from pathlib import Path +from typing import Protocol from PIL import Image from matteloop.core.crop import clamp_crop -from matteloop.core.errors import ErrorCode, ValidationError +from matteloop.core.exclusion import exclude_alpha +from matteloop.core.fingerprints import cut_union_fingerprint from matteloop.core.geometry import FramingPlan, PixelBounds, apply_framing from matteloop.core.rgba import RgbaOwnershipTracker -from matteloop.core.specs import FramingSpec, TransformSpec +from matteloop.core.specs import CropSpec, FramingSpec, TransformSpec from matteloop.core.tokens import ProgressStage from matteloop.core.transform import apply_transform from matteloop.jobs.context import JobContext from matteloop.jobs.encoding import _map_output_os_error, _output_error +from matteloop.jobs.workspace import CutManifest, CutUnionMetadata, CutWorkspace + + +class _RebuildWorkspace(Protocol): + def read_cut( + self, workspace: CutWorkspace, index: int, ownership: RgbaOwnershipTracker + ) -> Image.Image: ... + + +def cached_union( + metadata: CutUnionMetadata | None, + framing: FramingSpec, + crop: CropSpec, + cut_key: str, +) -> PixelBounds | None: + """Return cached bounds only when their effective inputs still match.""" + if metadata is None or metadata.fingerprint != cut_union_fingerprint( + framing, crop, cut_key=cut_key + ): + return None + return PixelBounds(*metadata.bounds) + + +def _iter_rebuild_frames( + workspace: _RebuildWorkspace, private: CutWorkspace, manifest: CutManifest, + tracker: RgbaOwnershipTracker, context: JobContext, boxes: tuple[PixelBounds, ...], +) -> Iterator[Image.Image]: + for index in range(manifest.frame_count): + context.checkpoint("rebuild-union") + with closing(workspace.read_cut(private, index, tracker)) as image: + exclude_alpha(image, boxes) + yield image def framing_plan( source_size: tuple[int, int], union: PixelBounds | None, framing: FramingSpec ) -> FramingPlan: """Build the one immutable ``FramingPlan`` the encoder and the player share.""" - if framing.trim and union is None: - raise ValidationError( - ErrorCode.INVALID_FRAMING, - "framing", - "range-wide alpha union contains no visible pixels at this threshold", - ) return FramingPlan( source_size, global_bounds=union if framing.trim else None, @@ -61,6 +90,8 @@ def stage_encoder_frames( tracker: RgbaOwnershipTracker, context: JobContext, overall: tuple[int, int], + *, + exclusions: tuple[PixelBounds, ...] = (), ) -> tuple[tuple[Path, ...], tuple[int, ...]]: """Frame, crop, and resize the kept cut frames and persist them. @@ -88,6 +119,31 @@ def stage_encoder_frames( raise _map_output_os_error( error, "cannot create framed input directory" ) from error + framed_paths = _stage_kept_frames( + read_cut, + frame_count, + plan, + transform, + framed_directory, + tracker, + context, + overall, + exclusions, + ) + return framed_paths, transform.select_kept(delays) + + +def _stage_kept_frames( + read_cut: Callable[[int, RgbaOwnershipTracker], Image.Image], + frame_count: int, + plan: FramingPlan, + transform: TransformSpec, + framed_directory: Path, + tracker: RgbaOwnershipTracker, + context: JobContext, + overall: tuple[int, int], + exclusions: tuple[PixelBounds, ...], +) -> tuple[Path, ...]: kept = transform.kept_range(frame_count) kept_count = len(kept) framed_paths: list[Path] = [] @@ -96,7 +152,14 @@ def stage_encoder_frames( context.checkpoint("framing") framed_paths.append( _stage_frame( - read_cut, index, position, plan, transform, framed_directory, tracker + read_cut, + index, + position, + plan, + transform, + framed_directory, + tracker, + exclusions, ) ) context.progress( @@ -107,7 +170,7 @@ def stage_encoder_frames( overall_completed=overall[0] + position + 1, overall_total=overall[1], ) - return tuple(framed_paths), transform.select_kept(delays) + return tuple(framed_paths) def _stage_frame( @@ -118,9 +181,11 @@ def _stage_frame( transform: TransformSpec, framed_directory: Path, tracker: RgbaOwnershipTracker, + exclusions: tuple[PixelBounds, ...], ) -> Path: cut = read_cut(index, tracker) try: + exclude_alpha(cut, exclusions) framed = apply_framing(cut, plan) tracker.register(framed) finally: diff --git a/src/matteloop/jobs/workspace/_manifest_validation.py b/src/matteloop/jobs/workspace/_manifest_validation.py index 2c51381..000e5f8 100644 --- a/src/matteloop/jobs/workspace/_manifest_validation.py +++ b/src/matteloop/jobs/workspace/_manifest_validation.py @@ -74,18 +74,21 @@ def _frame_from_payload(value: object) -> CutFrame: def _validate_cache_inputs(inputs: FrozenJsonMap) -> None: + expected_keys = { + "crop", + "edge_settings", + "model", + "orientation_color_version", + "pipeline_schema_version", + "rembg_version", + "sampling", + "source_sha256", + } + if "model_exclusions" in inputs: + expected_keys.add("model_exclusions") _exact_keys( inputs, - { - "crop", - "edge_settings", - "model", - "orientation_color_version", - "pipeline_schema_version", - "rembg_version", - "sampling", - "source_sha256", - }, + expected_keys, "cache_key_inputs", ) _validate_sha256(_string(inputs["source_sha256"], "source sha256"), "source sha256") @@ -151,6 +154,51 @@ def _validate_cache_inputs(inputs: FrozenJsonMap) -> None: minimum=0, maximum=_MAX_INT64, ) + if "model_exclusions" in inputs: + _validate_model_exclusions(inputs["model_exclusions"]) + + +def _validate_model_exclusions(value: FrozenJsonValue) -> None: + exclusions = _frozen_object(value, "model exclusions") + _exact_keys(exclusions, {"boxes", "fill"}, "model exclusions") + _bounded_text( + _string(exclusions["fill"], "model exclusion fill"), + "model exclusion fill", + ) + boxes = exclusions["boxes"] + if type(boxes) is not tuple: + raise _manifest_error("model exclusion boxes must be an array") + for box in boxes: + if type(box) is not tuple or len(box) != 4: + raise _manifest_error("model exclusion boxes must contain four integers") + left = _bounded_int( + _int(box[0], "model exclusion left"), + "model exclusion left", + minimum=0, + maximum=MAX_CUT_DIMENSION, + ) + top = _bounded_int( + _int(box[1], "model exclusion top"), + "model exclusion top", + minimum=0, + maximum=MAX_CUT_DIMENSION, + ) + right = _bounded_int( + _int(box[2], "model exclusion right"), + "model exclusion right", + minimum=0, + maximum=MAX_CUT_DIMENSION, + ) + bottom = _bounded_int( + _int(box[3], "model exclusion bottom"), + "model exclusion bottom", + minimum=0, + maximum=MAX_CUT_DIMENSION, + ) + if right <= left or bottom <= top: + raise _manifest_error( + "model exclusion boxes must be positive rectangles" + ) def _fraction_payload(value: FrozenJsonValue, field: str) -> float: diff --git a/src/matteloop/ui/controller.py b/src/matteloop/ui/controller.py index 724b3b4..ecd9573 100644 --- a/src/matteloop/ui/controller.py +++ b/src/matteloop/ui/controller.py @@ -441,6 +441,7 @@ def _source_loaded( _as_app_error(TypeError("source adapter returned an invalid result")), ) return + before = self._store.state self._store.dispatch( SourceLoaded( source_id, @@ -449,6 +450,9 @@ def _source_loaded( loaded.frame, ) ) + after = self._store.state + if after is not before and self._settings is not None: + persist_parameters(self._settings, after.parameters) @Slot(str, str, object) def _source_failed(self, source_id: str, request_id: str, error: object) -> None: diff --git a/src/matteloop/ui/copy.py b/src/matteloop/ui/copy.py index a17b9bd..d585ffe 100644 --- a/src/matteloop/ui/copy.py +++ b/src/matteloop/ui/copy.py @@ -643,6 +643,7 @@ def inspector_label(value: str) -> str: "Horizontal stretch": QCoreApplication.translate( "Inspector", "Horizontal stretch" ), + "Exclusions": QCoreApplication.translate("Inspector", "Exclusions"), "Directory": QCoreApplication.translate("Inspector", "Directory"), "Filename": QCoreApplication.translate("Inspector", "Filename"), "Maximum size": QCoreApplication.translate("Inspector", "Maximum size"), diff --git a/src/matteloop/ui/crop_canvas.py b/src/matteloop/ui/crop_canvas.py index 75e915b..2caf1dd 100644 --- a/src/matteloop/ui/crop_canvas.py +++ b/src/matteloop/ui/crop_canvas.py @@ -84,6 +84,9 @@ def apply_presentation( active: bool, editable: bool, ) -> None: + if presentation is None: + self._dragged = None + self._drag_crop = None self._presentation = presentation self._active = active self._editable = editable @@ -99,17 +102,29 @@ def apply_presentation( def paintEvent(self, event) -> None: # type: ignore[no-untyped-def] super().paintEvent(event) - if not self._active or self._geometry is None: + if self._geometry is None or self._presentation is None: + return + if not self._active: return geometry = self._geometry transform = geometry.transform if not isinstance(transform, MediaTransform): return content = transform.content_rect - crop = _qt_rect(geometry.visual["crop"]) + crop = self._widget_rect(geometry, self._presentation.crop) overlay = QColor(CANVAS_COLOR) overlay.setAlpha(105) painter = QPainter(self) + self._paint_crop_mask(painter, content, crop, overlay) + self._paint_crop_outline(painter, crop) + self._paint_handles(painter, geometry) + self._paint_focus_ring(painter, geometry) + painter.end() + + @staticmethod + def _paint_crop_mask( + painter: QPainter, content: RectF, crop: QRectF, overlay: QColor + ) -> None: painter.fillRect( QRectF(content.x, content.y, content.width, crop.top() - content.y), overlay, @@ -136,19 +151,32 @@ def paintEvent(self, event) -> None: # type: ignore[no-untyped-def] ), overlay, ) + + def _paint_crop_outline(self, painter: QPainter, crop: QRectF) -> None: + painter.setBrush(Qt.BrushStyle.NoBrush) painter.setPen(QPen(QColor(ACCENT_COLOR), 2)) painter.drawRect(crop) + + @staticmethod + def _paint_handles( + painter: QPainter, geometry: InteractionGeometry + ) -> None: painter.setPen(Qt.PenStyle.NoPen) painter.setBrush(QColor(ACCENT_COLOR)) for name in _HANDLE_NAMES: painter.drawRect(_qt_rect(geometry.visual[name])) - if self.hasFocus(): - focused = geometry.focus.get(self._focused_target) - if focused is not None: - painter.setBrush(Qt.BrushStyle.NoBrush) - painter.setPen(QPen(QColor(TEXT_COLOR), 1, Qt.PenStyle.DashLine)) - painter.drawRect(_qt_rect(focused).adjusted(-3, -3, 3, 3)) - painter.end() + + def _paint_focus_ring( + self, painter: QPainter, geometry: InteractionGeometry + ) -> None: + if not self.hasFocus(): + return + focused = geometry.focus.get(self._focused_target) + if focused is None: + return + painter.setBrush(Qt.BrushStyle.NoBrush) + painter.setPen(QPen(QColor(TEXT_COLOR), 1, Qt.PenStyle.DashLine)) + painter.drawRect(_qt_rect(focused).adjusted(-3, -3, 3, 3)) def mousePressEvent(self, event: QMouseEvent) -> None: if ( @@ -206,9 +234,10 @@ def keyPressEvent(self, event: QKeyEvent) -> None: super().keyPressEvent(event) return step = 10 if event.modifiers() & Qt.KeyboardModifier.ShiftModifier else 1 + edited = self._edited_rect() try: crop = nudge_crop( - self._presentation.crop, + edited, self._focused_target, dx=delta[0] * step, dy=delta[1] * step, @@ -219,7 +248,7 @@ def keyPressEvent(self, event: QKeyEvent) -> None: super().keyPressEvent(event) return crop = self._constrain(crop, self._focused_target) - if crop != self._presentation.crop: + if crop != edited: self.command_requested.emit(self._crop_event(crop)) self._announce_crop() event.accept() @@ -258,6 +287,28 @@ def _emit_drag(self, position: QPointF) -> None: if self._presentation is None or crop != self._presentation.crop: self.command_requested.emit(self._crop_event(crop)) + def _edited_rect(self) -> CropSpec: + return ( + self._presentation.crop + if self._presentation is not None + else CropSpec(0, 0, 1, 1) + ) + + def _widget_rect( + self, geometry: InteractionGeometry, rect: CropSpec + ) -> QRectF: + presentation = self._presentation + if presentation is None or not isinstance(geometry.transform, MediaTransform): + return QRectF() + raw = oriented_rect_to_source_rect( + rect, + source_width=presentation.coded_width, + source_height=presentation.coded_height, + rotation=presentation.rotation, + pixel_aspect=presentation.pixel_aspect, + ) + return _qt_rect(geometry.transform.source_rect_to_widget(raw)) + def _constrain(self, crop: CropSpec, target: str) -> CropSpec: """Re-fit a candidate crop before it is compared/emitted. @@ -286,7 +337,7 @@ def _rebuild_geometry( return try: raw_crop = oriented_rect_to_source_rect( - presentation.crop, + self._edited_rect(), source_width=presentation.coded_width, source_height=presentation.coded_height, rotation=presentation.rotation, @@ -328,7 +379,7 @@ def _announce_crop(self) -> None: presentation = self._presentation if presentation is None: return - crop = presentation.crop + crop = self._edited_rect() self.setAccessibleDescription( QCoreApplication.translate( "CropCanvas", diff --git a/src/matteloop/ui/crop_presentation.py b/src/matteloop/ui/crop_presentation.py index 023ae33..1a3bc2a 100644 --- a/src/matteloop/ui/crop_presentation.py +++ b/src/matteloop/ui/crop_presentation.py @@ -18,6 +18,8 @@ class CropPresentation: rotation: int pixel_aspect: float crop: CropSpec + exclusions: tuple[CropSpec, ...] = () + exclusions_before_model: bool = False def present_crop(state: AppState) -> CropPresentation | None: @@ -60,4 +62,6 @@ def present_crop(state: AppState) -> CropPresentation | None: rotation, aspect, state.crop, + state.parameters.exclusions, + state.parameters.exclusions_before_model, ) diff --git a/src/matteloop/ui/exclusion_canvas.py b/src/matteloop/ui/exclusion_canvas.py new file mode 100644 index 0000000..6f8bc90 --- /dev/null +++ b/src/matteloop/ui/exclusion_canvas.py @@ -0,0 +1,790 @@ +"""Region-mode extension for the source crop canvas.""" + +from __future__ import annotations + +from pathlib import Path + +from PySide6.QtCore import QCoreApplication, QPoint, QPointF, QRectF, Qt, Signal +from PySide6.QtGui import ( + QBrush, + QColor, + QContextMenuEvent, + QKeyEvent, + QMouseEvent, + QPainter, + QPainterPath, + QPen, +) +from PySide6.QtWidgets import QMenu, QWidget + +from matteloop.core.crop import ( + crop_from_drag, + nudge_crop, + oriented_point_from_widget, +) +from matteloop.core.exclusion import cut_exclusions +from matteloop.core.geometry import InteractionGeometry, MediaTransform, PointF +from matteloop.core.parameters import ExclusionsChanged +from matteloop.core.specs import CropSpec +from matteloop.ui.crop_canvas import ( + _HANDLE_NAMES, + CropCanvas, + _key_delta, +) +from matteloop.ui.crop_presentation import CropPresentation +from matteloop.ui.exclusion_context_menu import ( + build_context_menu, + contains, + context_menu_position, + exclusion_at, + handle_context_menu_event, + handle_context_menu_key, + remove_all_exclusions, + remove_exclusion, + selected_context_region, +) +from matteloop.ui.preview_canvas import PreviewCanvas +from matteloop.ui.theme import ACCENT_COLOR, CANVAS_COLOR + + +class ExclusionCanvas(CropCanvas): + """Add source-region editing while retaining CropCanvas's crop hooks.""" + + exclusion_edit_toggled = Signal(bool) + exclusion_selection_changed = Signal(object, object) + + def __init__( + self, + parent: QWidget | None = None, + *, + title: str = "Original", + object_name: str = "original_canvas", + runtime_root: Path | None = None, + ) -> None: + super().__init__( + parent, + title=title, + object_name=object_name, + runtime_root=runtime_root, + ) + self._exclusion_edit = False + self._selected_exclusion: int | None = None + self._selected_exclusion_value: CropSpec | None = None + self._drag_exclusion: int | None = None + self._drag_origin_region: CropSpec | None = None + self._drag_region: CropSpec | None = None + self._rubber_start: PointF | None = None + self._rubber_current: PointF | None = None + + def set_exclusion_edit(self, enabled: bool) -> None: + """Toggle the mutually exclusive exclusion-region editing mode.""" + if self._exclusion_edit == enabled: + return + self._exclusion_edit = enabled + self._clear_exclusion_selection() + self._clear_gesture() + self._focused_target = "crop" + self._rebuild_geometry() + self._announce_crop() + self.update() + self.exclusion_edit_toggled.emit(enabled) + + def contextMenuEvent(self, event: QContextMenuEvent) -> None: + handle_context_menu_event(self, event) + + def _build_context_menu( + self, position: QPoint, *, target: CropSpec | None = None + ) -> QMenu: + return build_context_menu(self, position, target=target) + + def _context_menu_position(self, event: QContextMenuEvent) -> QPoint: + return context_menu_position(self, event) + + def _selected_context_region(self) -> CropSpec | None: + return selected_context_region(self) + + def _show_context_menu(self, menu: QMenu, position: QPoint) -> None: + menu.exec(self.mapToGlobal(position)) + + def _exclusion_at(self, position: QPoint) -> int | None: + return exclusion_at(self, position) + + def _remove_exclusion(self, target: CropSpec) -> None: + remove_exclusion(self, target) + + def _remove_all_exclusions(self) -> None: + remove_all_exclusions(self) + + def apply_presentation( + self, + presentation: CropPresentation | None, + *, + active: bool, + editable: bool, + ) -> None: + previous = self._presentation + if presentation is None: + self._clear_exclusion_selection() + self._clear_gesture() + else: + self._reconcile_exclusion_state(previous, presentation) + super().apply_presentation( + presentation, active=active, editable=editable + ) + if presentation is None: + self._emit_exclusion_selection() + + def paintEvent(self, event) -> None: # type: ignore[no-untyped-def] + PreviewCanvas.paintEvent(self, event) + if self._geometry is None or self._presentation is None: + return + if not ( + self._active + or self._exclusion_edit + or self._presentation.exclusions + ): + return + geometry = self._geometry + transform = geometry.transform + if not isinstance(transform, MediaTransform): + return + content = transform.content_rect + crop = self._widget_rect(geometry, self._presentation.crop) + overlay = QColor(CANVAS_COLOR) + overlay.setAlpha(105) + painter = QPainter(self) + if self._active: + self._paint_crop_mask(painter, content, crop, overlay) + self._paint_exclusions(painter, geometry) + if self._active: + self._paint_crop_outline(painter, crop) + show_controls = ( + self._active and not self._exclusion_edit + ) or (self._exclusion_edit and self._selected_exclusion is not None) + if show_controls: + self._paint_handles(painter, geometry) + if show_controls: + self._paint_focus_ring(painter, geometry) + painter.end() + + def _paint_crop_outline(self, painter: QPainter, crop: QRectF) -> None: + color = QColor(ACCENT_COLOR) + if self._exclusion_edit: + color.setAlpha(135) + painter.setBrush(Qt.BrushStyle.NoBrush) + painter.setPen(QPen(color, 1 if self._exclusion_edit else 2)) + painter.drawRect(crop) + + def _paint_exclusions( + self, painter: QPainter, geometry: InteractionGeometry + ) -> None: + effective_brush, effective_pen = self._effective_exclusion_style() + ineffective_fill = QColor("#E5484D") + ineffective_fill.setAlpha(32) + ineffective_pen = QPen( + QColor("#E5484D"), 1, Qt.PenStyle.DashLine + ) + for exclusion in self._painted_exclusions(): + region_rect = self._widget_rect(geometry, exclusion) + effective = self._effective_exclusion(exclusion) + if effective is None: + self._paint_exclusion_rect( + painter, + region_rect, + fill=ineffective_fill, + pen=ineffective_pen, + ) + continue + effective_rect = self._widget_rect(geometry, effective) + if effective == exclusion: + self._paint_exclusion_rect( + painter, + effective_rect, + fill=effective_brush, + pen=effective_pen, + ) + continue + region_path = QPainterPath() + region_path.addRect(region_rect) + effective_path = QPainterPath() + effective_path.addRect(effective_rect) + painter.fillPath( + region_path.subtracted(effective_path), ineffective_fill + ) + self._paint_exclusion_rect( + painter, region_rect, fill=None, pen=ineffective_pen + ) + self._paint_exclusion_rect( + painter, + effective_rect, + fill=effective_brush, + pen=effective_pen, + ) + + def _effective_exclusion_style(self) -> tuple[QColor | QBrush, QPen]: + effective_fill = QColor("#E5484D") + effective_fill.setAlpha(90) + effective_pen = QPen(QColor("#E5484D"), 1) + presentation = self._presentation + if presentation is not None and presentation.exclusions_before_model: + effective_brush: QColor | QBrush = QBrush( + effective_fill, Qt.BrushStyle.BDiagPattern + ) + effective_pen = QPen(QColor("#E5484D"), 2) + else: + effective_brush = effective_fill + return effective_brush, effective_pen + + @staticmethod + def _paint_exclusion_rect( + painter: QPainter, + rect: QRectF, + *, + fill: QColor | QBrush | None, + pen: QPen, + ) -> None: + painter.setBrush(fill if fill is not None else Qt.BrushStyle.NoBrush) + painter.setPen(pen) + painter.drawRect(rect) + + def _effective_exclusion(self, exclusion: CropSpec) -> CropSpec | None: + if self._presentation is None: + return None + boxes = cut_exclusions((exclusion,), self._presentation.crop) + if not boxes: + return None + box = boxes[0] + crop = self._presentation.crop + return CropSpec( + crop.x + box.left, + crop.y + box.top, + box.width, + box.height, + ) + + def mousePressEvent(self, event: QMouseEvent) -> None: + if not self._exclusion_edit: + super().mousePressEvent(event) + return + if ( + not self._editable + or (not self._active and not self._exclusion_edit) + or event.button() != Qt.MouseButton.LeftButton + ): + super().mousePressEvent(event) + return + geometry = self._geometry + if geometry is None: + super().mousePressEvent(event) + return + self._press_exclusion( + PointF(event.position().x(), event.position().y()) + ) + event.accept() + + def mouseMoveEvent(self, event: QMouseEvent) -> None: + if not self._exclusion_edit: + super().mouseMoveEvent(event) + return + if ( + self._drag_exclusion is not None + and self._dragged is not None + and self._editable + and event.buttons() & Qt.MouseButton.LeftButton + ): + self._emit_exclusion_drag(event.position()) + event.accept() + return + if ( + self._rubber_start is not None + and event.buttons() & Qt.MouseButton.LeftButton + ): + self._rubber_current = self._oriented_position(event.position()) + self.update() + event.accept() + return + super().mouseMoveEvent(event) + + def mouseReleaseEvent(self, event: QMouseEvent) -> None: + if not self._exclusion_edit: + super().mouseReleaseEvent(event) + return + if event.button() == Qt.MouseButton.LeftButton: + if self._drag_exclusion is not None: + self._finish_exclusion_drag() + event.accept() + return + if self._rubber_start is not None: + self._finish_rubber_band() + event.accept() + return + super().mouseReleaseEvent(event) + + def keyPressEvent(self, event: QKeyEvent) -> None: + if self._handle_context_menu_key(event): + return + if not self._exclusion_edit: + super().keyPressEvent(event) + return + if not self._editable or not self.hasFocus(): + super().keyPressEvent(event) + return + if self._handle_exclusion_key(event): + return + super().keyPressEvent(event) + + def _handle_context_menu_key(self, event: QKeyEvent) -> bool: + return handle_context_menu_key(self, event) + + def _handle_exclusion_key(self, event: QKeyEvent) -> bool: + if event.key() in {int(Qt.Key.Key_Delete), int(Qt.Key.Key_Backspace)}: + if self._selected_exclusion is None: + return False + self._delete_selected_exclusion(event) + return True + if event.key() == int(Qt.Key.Key_Escape): + self._clear_exclusion_selection() + self._clear_gesture() + self._rebuild_geometry() + self._announce_crop() + self.update() + event.accept() + return True + if event.key() in { + int(Qt.Key.Key_BracketLeft), + int(Qt.Key.Key_BracketRight), + }: + if self._cycle_exclusion_selection( + backwards=event.key() == int(Qt.Key.Key_BracketLeft) + ): + event.accept() + return True + return False + if event.key() in { + int(Qt.Key.Key_Insert), + int(Qt.Key.Key_N), + int(Qt.Key.Key_Space), + }: + self._create_exclusion() + event.accept() + return True + if self._selected_exclusion is None: + return False + delta = _key_delta(event) + if delta is None or self._presentation is None: + return False + step = 10 if event.modifiers() & Qt.KeyboardModifier.ShiftModifier else 1 + edited = self._edited_rect() + try: + region = nudge_crop( + edited, + self._focused_target, + dx=delta[0] * step, + dy=delta[1] * step, + source_width=self._presentation.width, + source_height=self._presentation.height, + ) + except ValueError: + return False + region = self._constrain(region, self._focused_target) + if region != edited: + self._set_exclusion_selection( + self._selected_exclusion, value=region + ) + self.command_requested.emit(self._crop_event(region)) + self._announce_crop() + event.accept() + return True + + def _delete_selected_exclusion(self, event: QKeyEvent) -> None: + index = self._selected_exclusion + if self._presentation is None or index is None: + super().keyPressEvent(event) + return + current = self._presentation.exclusions + exclusions = current[:index] + current[index + 1 :] + self._clear_exclusion_selection() + self._clear_gesture() + self.command_requested.emit(ExclusionsChanged(exclusions)) + self._rebuild_geometry() + self._announce_crop() + self.update() + event.accept() + + def _press_exclusion(self, position: PointF) -> None: + if self._presentation is None or self._geometry is None: + return + selected = self._selected_exclusion + if selected is not None: + target = self._geometry.hit_test(position) + if target in {"crop", *_HANDLE_NAMES}: + self._begin_exclusion_drag(selected, target, position) + return + oriented = self._oriented_position(position) + for index in range(len(self._presentation.exclusions) - 1, -1, -1): + if index == selected: + continue + if contains(self._presentation.exclusions[index], oriented): + self._set_exclusion_selection(index) + self._focused_target = "crop" + self._announce_crop() + self._begin_exclusion_drag(index, "crop", position) + return + self._clear_gesture() + self._rubber_start = self._clamp_oriented(oriented) + self._rubber_current = self._rubber_start + self.setFocus(Qt.FocusReason.MouseFocusReason) + self.update() + + def _begin_exclusion_drag( + self, index: int, target: str, position: PointF + ) -> None: + if self._presentation is None or self._geometry is None: + return + self._drag_exclusion = index + self._drag_origin_region = self._presentation.exclusions[index] + self._drag_region = self._drag_origin_region + self._dragged = target + self._drag_start = self._oriented_position(position) + self.setFocus(Qt.FocusReason.MouseFocusReason) + self._rebuild_geometry(focused=target, dragged=target) + self.update() + + def _emit_exclusion_drag(self, position: QPointF) -> None: + if ( + self._geometry is None + or self._drag_origin_region is None + or self._dragged is None + ): + return + current = self._oriented_position(position) + region = crop_from_drag( + self._drag_origin_region, + self._dragged, + self._drag_start, + current, + source_width=self._presentation.width if self._presentation else 1, + source_height=self._presentation.height if self._presentation else 1, + ) + region = self._constrain(region, self._dragged) + if region == self._drag_region: + return + self._drag_region = region + self._rebuild_geometry(focused=self._dragged, dragged=self._dragged) + self._announce_crop() + self.update() + + def _finish_exclusion_drag(self) -> None: + index = self._drag_exclusion + region = self._drag_region + original = ( + self._presentation.exclusions[index] + if self._presentation is not None + and index is not None + and index < len(self._presentation.exclusions) + else None + ) + self._clear_gesture() + if index is not None and region is not None: + self._set_exclusion_selection(index, value=region) + if region != original: + self.command_requested.emit(self._exclusion_event(region)) + self._rebuild_geometry() + self._announce_crop() + self.update() + + def _finish_rubber_band(self) -> None: + region = self._rubber_region() + self._clear_gesture() + if ( + region is not None + and region.width >= 8 + and region.height >= 8 + and self._presentation is not None + ): + self._set_exclusion_selection( + len(self._presentation.exclusions), value=region + ) + self.command_requested.emit(self._exclusion_event(region)) + self._rebuild_geometry() + self._announce_crop() + self.update() + + def _exclusion_event(self, region: CropSpec) -> ExclusionsChanged: + if self._presentation is None or self._selected_exclusion is None: + return ExclusionsChanged((region,)) + exclusions = list(self._presentation.exclusions) + if self._selected_exclusion == len(exclusions): + exclusions.append(region) + elif self._selected_exclusion > len(exclusions): + return ExclusionsChanged(tuple(exclusions)) + else: + exclusions[self._selected_exclusion] = region + return ExclusionsChanged(tuple(exclusions)) + + def _oriented_position(self, position: QPoint | QPointF | PointF) -> PointF: + if self._geometry is None: + return PointF(0, 0) + point = ( + position + if isinstance(position, PointF) + else PointF(position.x(), position.y()) + ) + return oriented_point_from_widget(self._geometry, point) + + def _clamp_oriented(self, point: PointF) -> PointF: + if self._presentation is None: + return point + return PointF( + min(max(point.x, 0), self._presentation.width), + min(max(point.y, 0), self._presentation.height), + ) + + def _rubber_region(self) -> CropSpec | None: + if self._rubber_start is None or self._rubber_current is None: + return None + start = self._clamp_oriented(self._rubber_start) + current = self._clamp_oriented(self._rubber_current) + left, right = sorted((round(start.x), round(current.x))) + top, bottom = sorted((round(start.y), round(current.y))) + if right <= left or bottom <= top: + return None + return CropSpec(left, top, right - left, bottom - top) + + def _clear_gesture(self) -> None: + self._dragged = None + self._drag_crop = None + self._drag_exclusion = None + self._drag_origin_region = None + self._drag_region = None + self._rubber_start = None + self._rubber_current = None + + def _clear_exclusion_selection(self) -> None: + self._selected_exclusion = None + self._selected_exclusion_value = None + + def _set_exclusion_selection( + self, index: int, *, value: CropSpec | None = None + ) -> None: + if self._selected_exclusion != index: + self._clear_gesture() + self._selected_exclusion = index + if value is not None: + self._selected_exclusion_value = value + elif self._presentation is not None and index < len( + self._presentation.exclusions + ): + self._selected_exclusion_value = self._presentation.exclusions[index] + else: + self._selected_exclusion_value = None + + def _cycle_exclusion_selection(self, *, backwards: bool) -> bool: + if self._presentation is None or not self._presentation.exclusions: + return False + self._clear_gesture() + count = len(self._presentation.exclusions) + current = self._selected_exclusion + if current is None or current >= count: + index = count - 1 if backwards else 0 + else: + index = (current - 1 if backwards else current + 1) % count + self._set_exclusion_selection(index) + self._focused_target = "crop" + self._rebuild_geometry() + self._announce_crop() + self.update() + return True + + def _create_exclusion(self, position: QPoint | None = None) -> None: + if self._presentation is None or not self._editable: + return + if not self._exclusion_edit: + self.set_exclusion_edit(True) + width = max(1, round(self._presentation.width * 0.2)) + height = max(1, round(self._presentation.height * 0.2)) + width = min(width, self._presentation.width) + height = min(height, self._presentation.height) + if position is None: + center = PointF( + self._presentation.width / 2, self._presentation.height / 2 + ) + else: + center = self._clamp_oriented(self._oriented_position(position)) + left = min( + max(round(center.x - width / 2), 0), self._presentation.width - width + ) + top = min( + max(round(center.y - height / 2), 0), self._presentation.height - height + ) + region = CropSpec( + left, + top, + width, + height, + ) + self._set_exclusion_selection( + len(self._presentation.exclusions), value=region + ) + self.setFocus(Qt.FocusReason.OtherFocusReason) + self._rebuild_geometry() + self._announce_crop() + self.update() + self.command_requested.emit(self._exclusion_event(region)) + + def _reconcile_exclusion_state( + self, + previous: CropPresentation | None, + presentation: CropPresentation, + ) -> None: + source_changed = previous is None or ( + previous.source_id != presentation.source_id + or previous.width != presentation.width + or previous.height != presentation.height + or previous.rotation != presentation.rotation + or previous.pixel_aspect != presentation.pixel_aspect + ) + if self._drag_exclusion is not None: + drag_index = self._drag_exclusion + if ( + source_changed + or self._drag_origin_region is None + or drag_index >= len(presentation.exclusions) + or presentation.exclusions[drag_index] != self._drag_origin_region + ): + self._clear_gesture() + elif self._rubber_start is not None and ( + source_changed + or previous is None + or previous.exclusions != presentation.exclusions + ): + self._clear_gesture() + index = self._selected_exclusion + if index is None: + return + value = self._selected_exclusion_value + if value is None and previous is not None and index < len( + previous.exclusions + ): + value = previous.exclusions[index] + if value is None: + self._clear_exclusion_selection() + return + if ( + index < len(presentation.exclusions) + and presentation.exclusions[index] == value + ): + self._selected_exclusion_value = value + return + try: + self._selected_exclusion = presentation.exclusions.index(value) + except ValueError: + self._clear_exclusion_selection() + + def _painted_exclusions(self) -> tuple[CropSpec, ...]: + if self._presentation is None: + return () + exclusions = list(self._presentation.exclusions) + if ( + self._drag_exclusion is not None + and self._drag_region is not None + and self._drag_exclusion < len(exclusions) + ): + exclusions[self._drag_exclusion] = self._drag_region + rubber = self._rubber_region() + if rubber is not None: + exclusions.append(rubber) + return tuple(exclusions) + + def _edited_rect(self) -> CropSpec: + if self._presentation is None or not self._exclusion_edit: + return super()._edited_rect() + index = self._selected_exclusion + if index is None or index >= len(self._presentation.exclusions): + return self._presentation.crop + return self._drag_region or self._presentation.exclusions[index] + + def _crop_event(self, crop: CropSpec) -> object: + if self._exclusion_edit: + return self._exclusion_event(crop) + return super()._crop_event(crop) + + def _announce_crop(self) -> None: + presentation = self._presentation + if presentation is None: + return + self._emit_exclusion_selection() + if self._exclusion_edit: + self._announce_exclusion_crop(presentation) + return + super()._announce_crop() + if presentation.exclusions_before_model: + self.setAccessibleDescription( + self.accessibleDescription() + + QCoreApplication.translate( + "CropCanvas", "; blanked before the model" + ) + ) + + def _announce_exclusion_crop(self, presentation: CropPresentation) -> None: + index = self._selected_exclusion + if index is None or index >= len(presentation.exclusions): + description = QCoreApplication.translate( + "CropCanvas", + "Exclusion region mode: %1 region(s); no region selected", + ).replace("%1", str(len(presentation.exclusions))) + if presentation.exclusions_before_model: + description += QCoreApplication.translate( + "CropCanvas", "; blanked before the model" + ) + self.setAccessibleDescription(description) + return + assert index is not None + crop = self._edited_rect() + effective = self._effective_exclusion(crop) + if effective is None: + text = QCoreApplication.translate( + "CropCanvas", + "Exclusion region %1 of %2: x %3, y %4, width %5, height %6 " + "source pixels; entirely outside crop", + ) + elif effective != crop: + text = QCoreApplication.translate( + "CropCanvas", + "Exclusion region %1 of %2: x %3, y %4, width %5, height %6 " + "source pixels; partly outside crop", + ) + else: + text = QCoreApplication.translate( + "CropCanvas", + "Exclusion region %1 of %2: x %3, y %4, width %5, height %6 " + "source pixels", + ) + values = ( + index + 1, + len(presentation.exclusions), + crop.x, + crop.y, + crop.width, + crop.height, + ) + for number, value in enumerate(values, start=1): + text = text.replace(f"%{number}", str(value)) + if presentation.exclusions_before_model: + text += QCoreApplication.translate( + "CropCanvas", "; blanked before the model" + ) + self.setAccessibleDescription(text) + + def _emit_exclusion_selection(self) -> None: + presentation = self._presentation + if presentation is None: + self.exclusion_selection_changed.emit(None, None) + return + index = self._selected_exclusion + exclusion = ( + presentation.exclusions[index] + if index is not None and index < len(presentation.exclusions) + else None + ) + self.exclusion_selection_changed.emit(exclusion, presentation.crop) diff --git a/src/matteloop/ui/exclusion_context_menu.py b/src/matteloop/ui/exclusion_context_menu.py new file mode 100644 index 0000000..a9b19a8 --- /dev/null +++ b/src/matteloop/ui/exclusion_context_menu.py @@ -0,0 +1,227 @@ +"""Context-menu behavior for the source exclusion canvas.""" + +from __future__ import annotations + +from collections.abc import Callable +from typing import TYPE_CHECKING + +from PySide6.QtCore import QCoreApplication, QPoint, Qt +from PySide6.QtGui import QAction, QContextMenuEvent, QKeyEvent +from PySide6.QtWidgets import QMenu + +from matteloop.core.geometry import MediaTransform, PointF +from matteloop.core.parameters import ExclusionsChanged +from matteloop.core.specs import CropSpec + +if TYPE_CHECKING: + from matteloop.ui.exclusion_canvas import ExclusionCanvas + + +def handle_context_menu_event( + canvas: ExclusionCanvas, event: QContextMenuEvent +) -> None: + if canvas._presentation is None or not canvas._editable: + event.ignore() + return + position = context_menu_position(canvas, event) + target = ( + selected_context_region(canvas) + if event.reason() is QContextMenuEvent.Reason.Keyboard + else None + ) + menu = build_context_menu(canvas, position, target=target) + if not menu.actions(): + menu.deleteLater() + event.ignore() + return + try: + canvas._show_context_menu(menu, position) + finally: + menu.deleteLater() + event.accept() + + +def build_context_menu( + canvas: ExclusionCanvas, position: QPoint, *, target: CropSpec | None = None +) -> QMenu: + menu = QMenu(canvas) + presentation = canvas._presentation + if presentation is None or not canvas._editable: + return menu + if target is None: + index = exclusion_at(canvas, position) + target = presentation.exclusions[index] if index is not None else None + if target is None: + add_context_action( + menu, + QCoreApplication.translate("ExclusionCanvas", "Exclude this area"), + "exclude_area", + lambda: canvas._create_exclusion(position), + ) + else: + assert target is not None + + def remove_target() -> None: + remove_exclusion(canvas, target) + + add_context_action( + menu, + QCoreApplication.translate("ExclusionCanvas", "Remove this region"), + "remove_exclusion", + remove_target, + ) + if presentation.exclusions: + add_context_action( + menu, + QCoreApplication.translate("ExclusionCanvas", "Remove all regions"), + "remove_all_exclusions", + lambda: remove_all_exclusions(canvas), + ) + add_edit_context_action(canvas, menu) + return menu + + +def add_edit_context_action(canvas: ExclusionCanvas, menu: QMenu) -> None: + action = add_context_action( + menu, + QCoreApplication.translate("ExclusionCanvas", "Edit exclusion regions"), + "edit_exclusion_regions", + None, + ) + action.setCheckable(True) + action.setChecked(canvas._exclusion_edit) + action.triggered.connect(lambda enabled: toggle_exclusion_edit(canvas, enabled)) + + +def add_context_action( + menu: QMenu, + text: str, + object_name: str, + callback: Callable[[], None] | None, +) -> QAction: + action = QAction(text, menu) + action.setObjectName(object_name) + action.setToolTip(text) + action.setStatusTip(text) + if callback is not None: + + def trigger(_checked: bool = False) -> None: + callback() + + action.triggered.connect(trigger) + menu.addAction(action) + return action + + +def context_menu_position(canvas: ExclusionCanvas, event: QContextMenuEvent) -> QPoint: + if event.reason() is QContextMenuEvent.Reason.Keyboard: + region = selected_context_region(canvas) + if region is not None and canvas._geometry is not None: + rect = canvas._widget_rect(canvas._geometry, region) + return QPoint(round(rect.center().x()), round(rect.center().y())) + if canvas._geometry is not None and isinstance( + canvas._geometry.transform, MediaTransform + ): + content = canvas._geometry.transform.content_rect + return QPoint( + round(content.x + content.width / 2), + round(content.y + content.height / 2), + ) + return canvas.rect().center() + position = event.pos() + return position if canvas.rect().contains(position) else canvas.rect().center() + + +def selected_context_region(canvas: ExclusionCanvas) -> CropSpec | None: + index = canvas._selected_exclusion + if ( + canvas._presentation is None + or index is None + or index >= len(canvas._presentation.exclusions) + ): + return None + return canvas._presentation.exclusions[index] + + +def exclusion_at(canvas: ExclusionCanvas, position: QPoint) -> int | None: + if canvas._presentation is None: + return None + oriented = canvas._oriented_position(position) + for index in range(len(canvas._presentation.exclusions) - 1, -1, -1): + if contains(canvas._presentation.exclusions[index], oriented): + return index + return None + + +def remove_exclusion(canvas: ExclusionCanvas, target: CropSpec) -> None: + if canvas._presentation is None or not canvas._editable: + return + exclusions = canvas._presentation.exclusions + index = next( + (index for index, exclusion in enumerate(exclusions) if exclusion is target), + None, + ) + if index is None: + return + updated = exclusions[:index] + exclusions[index + 1 :] + if canvas._selected_exclusion == index: + canvas._clear_exclusion_selection() + elif ( + canvas._selected_exclusion is not None + and canvas._selected_exclusion > index + ): + canvas._selected_exclusion -= 1 + canvas._clear_gesture() + canvas._rebuild_geometry() + canvas._announce_crop() + canvas.update() + canvas.command_requested.emit(ExclusionsChanged(updated)) + + +def remove_all_exclusions(canvas: ExclusionCanvas) -> None: + if ( + canvas._presentation is None + or not canvas._editable + or not canvas._presentation.exclusions + ): + return + canvas._clear_exclusion_selection() + canvas._clear_gesture() + canvas._rebuild_geometry() + canvas._announce_crop() + canvas.update() + canvas.command_requested.emit(ExclusionsChanged(())) + + +def toggle_exclusion_edit(canvas: ExclusionCanvas, enabled: bool) -> None: + if canvas._presentation is not None and canvas._editable: + canvas.set_exclusion_edit(enabled) + + +def handle_context_menu_key(canvas: ExclusionCanvas, event: QKeyEvent) -> bool: + if not ( + event.key() == int(Qt.Key.Key_Menu) + or ( + event.key() == int(Qt.Key.Key_F10) + and bool(event.modifiers() & Qt.KeyboardModifier.ShiftModifier) + ) + ): + return False + if canvas._presentation is None or not canvas._editable: + return False + position = canvas.rect().center() + context_event = QContextMenuEvent( + QContextMenuEvent.Reason.Keyboard, + position, + canvas.mapToGlobal(position), + ) + canvas.contextMenuEvent(context_event) + event.accept() + return True + + +def contains(region: CropSpec, point: PointF) -> bool: + return ( + region.x <= point.x < region.x + region.width + and region.y <= point.y < region.y + region.height + ) diff --git a/src/matteloop/ui/exclusion_controls.py b/src/matteloop/ui/exclusion_controls.py new file mode 100644 index 0000000..73b39e9 --- /dev/null +++ b/src/matteloop/ui/exclusion_controls.py @@ -0,0 +1,199 @@ +"""Inspector controls for the source-frame exclusion-region editor.""" + +from __future__ import annotations + +from PySide6.QtCore import QCoreApplication, Qt, Signal +from PySide6.QtWidgets import ( + QCheckBox, + QHBoxLayout, + QLabel, + QSizePolicy, + QToolButton, + QVBoxLayout, + QWidget, +) + +from matteloop.core.exclusion import cut_exclusions +from matteloop.core.parameters import ( + ExclusionsBeforeModelChanged, + ExclusionsChanged, +) +from matteloop.core.specs import CropSpec +from matteloop.ui.parameter_presentation import ParameterPresentation + + +class ExclusionControls(QWidget): + """Small, reducer-facing control row for exclusion regions.""" + + edit_toggled = Signal(bool) + command_requested = Signal(object) + + def __init__(self, parent: QWidget | None = None) -> None: + super().__init__(parent) + self.setObjectName("exclusion_controls") + self._build_widgets() + self._build_layout() + self.edit_button.toggled.connect(self.edit_toggled) + self.clear_button.clicked.connect( + lambda: self.command_requested.emit(ExclusionsChanged(())) + ) + self.before_model_checkbox.toggled.connect( + lambda enabled: self.command_requested.emit( + ExclusionsBeforeModelChanged(enabled) + ) + ) + + def apply(self, presentation: ParameterPresentation, editable: bool) -> None: + """Render the region count and current source editability.""" + count = len(presentation.exclusions) + if presentation.exclusions_before_model: + count_text = self.tr( + "%n region(s), blanked before the model", "", count + ) + else: + count_text = self.tr("%n region(s)", "", count) + self.count_label.setText(count_text) + self.count_label.setAccessibleDescription(count_text) + available = editable and presentation.duration is not None + self.edit_button.setEnabled(available) + self.clear_button.setEnabled(available and count > 0) + self.before_model_checkbox.setEnabled(available and count > 0) + blocked = self.before_model_checkbox.blockSignals(True) + try: + self.before_model_checkbox.setChecked( + presentation.exclusions_before_model + ) + finally: + self.before_model_checkbox.blockSignals(blocked) + if not available and self.edit_button.isChecked(): + self.edit_button.setChecked(False) + + def set_selected_exclusion( + self, exclusion: CropSpec | None, crop: CropSpec | None + ) -> None: + """Render the selected stored rectangle and its effective crop status.""" + if exclusion is None or crop is None: + text = QCoreApplication.translate("Inspector", "No region selected") + else: + # One complete sentence per case rather than a stem plus appended + # fragments: a fragment forces every language into English word + # order, and a language that must lead with the qualifier or + # restructure the clause cannot translate it at all. + text = self._format_selection( + _selection_template(_crop_relation(exclusion, crop)), exclusion + ) + self.selected_readout.setText(text) + self.selected_readout.setToolTip(text) + self.selected_readout.setAccessibleDescription(text) + + @staticmethod + def _format_selection(template: str, exclusion: CropSpec) -> str: + values = (exclusion.x, exclusion.y, exclusion.width, exclusion.height) + for number, value in enumerate(values, start=1): + template = template.replace(f"%{number}", str(value)) + return template + + def tab_widgets(self) -> tuple[QWidget, ...]: + """Return the row's keyboard controls in visual order.""" + return ( + self.edit_button, + self.clear_button, + self.before_model_checkbox, + ) + + def _build_widgets(self) -> None: + self.edit_button = QToolButton() + self.edit_button.setObjectName("exclusion_edit") + self.edit_button.setCheckable(True) + self.edit_button.setText(QCoreApplication.translate("Inspector", "Edit")) + self.edit_button.setAccessibleName( + QCoreApplication.translate("Inspector", "Edit exclusion regions") + ) + self.edit_button.setToolTip( + QCoreApplication.translate("Inspector", "Edit exclusion regions") + ) + self.clear_button = QToolButton() + self.clear_button.setObjectName("exclusion_clear") + self.clear_button.setText(QCoreApplication.translate("Inspector", "Clear")) + self.clear_button.setAccessibleName( + QCoreApplication.translate("Inspector", "Clear exclusion regions") + ) + self.before_model_checkbox = QCheckBox( + QCoreApplication.translate("Inspector", "Blank before the model") + ) + self.before_model_checkbox.setObjectName("exclusion_before_model") + self.before_model_checkbox.setSizePolicy( + QSizePolicy.Policy.Ignored, QSizePolicy.Policy.Preferred + ) + cost = QCoreApplication.translate( + "Inspector", + "Re-segments the clip: the stored cut set is not reused", + ) + self.before_model_checkbox.setToolTip(cost) + self.before_model_checkbox.setAccessibleDescription(cost) + self.count_label = QLabel() + self.count_label.setObjectName("exclusion_count") + self.count_label.setProperty("secondary", True) + self.count_label.setAlignment( + Qt.AlignmentFlag.AlignLeft | Qt.AlignmentFlag.AlignVCenter + ) + self.count_label.setText(self.tr("%n region(s)", "", 0)) + self.count_label.setAccessibleDescription(self.count_label.text()) + self.selected_readout = QLabel() + self.selected_readout.setObjectName("exclusion_selected_readout") + self.selected_readout.setProperty("secondary", True) + self.selected_readout.setWordWrap(True) + self.selected_readout.setSizePolicy( + QSizePolicy.Policy.Ignored, QSizePolicy.Policy.Preferred + ) + self.selected_readout.setAccessibleName( + QCoreApplication.translate("Inspector", "Selected exclusion region") + ) + self.set_selected_exclusion(None, None) + self.edit_button.setEnabled(False) + self.clear_button.setEnabled(False) + self.before_model_checkbox.setEnabled(False) + + def _build_layout(self) -> None: + layout = QVBoxLayout(self) + layout.setContentsMargins(0, 0, 0, 0) + layout.setSpacing(4) + actions = QHBoxLayout() + actions.setContentsMargins(0, 0, 0, 0) + actions.setSpacing(8) + actions.addWidget(self.edit_button) + actions.addWidget(self.clear_button) + layout.addLayout(actions) + layout.addWidget(self.before_model_checkbox) + layout.addWidget(self.count_label) + layout.addWidget(self.selected_readout) + + +def _crop_relation(exclusion: CropSpec, crop: CropSpec) -> str: + """Say whether the stored rectangle is fully, partly, or not in the crop.""" + boxes = cut_exclusions((exclusion,), crop) + if not boxes: + return "outside" + box = boxes[0] + effective = CropSpec(crop.x + box.left, crop.y + box.top, box.width, box.height) + return "inside" if effective == exclusion else "partial" + + +def _selection_template(relation: str) -> str: + """Return one complete translatable sentence for each relation.""" + if relation == "outside": + return QCoreApplication.translate( + "Inspector", + "Selected region: x %1, y %2, width %3, height %4 source pixels;" + " entirely outside crop", + ) + if relation == "partial": + return QCoreApplication.translate( + "Inspector", + "Selected region: x %1, y %2, width %3, height %4 source pixels;" + " partly outside crop", + ) + return QCoreApplication.translate( + "Inspector", + "Selected region: x %1, y %2, width %3, height %4 source pixels", + ) diff --git a/src/matteloop/ui/inspector.py b/src/matteloop/ui/inspector.py index 44f7728..6d8b724 100644 --- a/src/matteloop/ui/inspector.py +++ b/src/matteloop/ui/inspector.py @@ -64,6 +64,7 @@ section_title, ) from matteloop.ui.crop_presentation import CropPresentation +from matteloop.ui.exclusion_controls import ExclusionControls from matteloop.ui.inspector_disclosure import configure_disclosure, read_bool from matteloop.ui.inspector_reset import action_availability, build_reset_button from matteloop.ui.parameter_presentation import ( @@ -307,6 +308,8 @@ def _build_cleanup_parameter_controls(self) -> None: self.stretch_spinbox.setAccessibleName( QCoreApplication.translate("Inspector", "Horizontal stretch") ) + self.exclusion_controls = ExclusionControls() + self.exclusion_controls.command_requested.connect(self.command_requested.emit) def _build_output_parameter_controls(self) -> None: self.output_directory_edit = compact_field(MiddleElidingLineEdit()) @@ -510,6 +513,7 @@ def apply_parameters( self.max_size_spinbox, ): widget.setEnabled(available) + self.exclusion_controls.apply(presentation, editable) clear_enabled, reset_enabled = action_availability(presentation) self.clear_output_directory_button.setEnabled(available and clear_enabled) self.reset_parameters_button.setEnabled(available and reset_enabled) @@ -769,6 +773,7 @@ def _cleanup_controls(self) -> QWidget: layout.addRow(self._form_label("Alpha threshold"), self.alpha_threshold_spinbox) layout.addRow(self._form_label("Padding"), self.padding_spinbox) layout.addRow(self._form_label("Horizontal stretch"), self.stretch_spinbox) + layout.addRow(self._form_label("Exclusions"), self.exclusion_controls) return controls def _output_controls(self) -> QWidget: @@ -826,6 +831,7 @@ def parameter_tab_widgets(self) -> tuple[QWidget, ...]: self.alpha_threshold_spinbox, self.padding_spinbox, self.stretch_spinbox, + *self.exclusion_controls.tab_widgets(), self.output_directory_button, self.clear_output_directory_button, self.output_filename_edit, @@ -846,6 +852,7 @@ def tab_widgets(self) -> tuple[QWidget, ...]: *self.crop_tab_widgets(), self.trim_checkbox, self.alpha_threshold_spinbox, self.padding_spinbox, self.stretch_spinbox, + *self.exclusion_controls.tab_widgets(), self.disclosures["transform"][0], *self.transform_group.tab_widgets(), self.disclosures["output"][0], diff --git a/src/matteloop/ui/inspector_reset.py b/src/matteloop/ui/inspector_reset.py index a28e39d..ad9aa43 100644 --- a/src/matteloop/ui/inspector_reset.py +++ b/src/matteloop/ui/inspector_reset.py @@ -24,6 +24,11 @@ def can_reset_parameters(presentation: ParameterPresentation) -> bool: (presentation.alpha_threshold, defaults.alpha_threshold), (presentation.padding, defaults.padding), (presentation.stretch_x, defaults.stretch_x), + (presentation.exclusions, defaults.exclusions), + ( + presentation.exclusions_before_model, + defaults.exclusions_before_model, + ), (presentation.max_mib, defaults.max_mib), ) ): diff --git a/src/matteloop/ui/main_window.py b/src/matteloop/ui/main_window.py index 3e6e0a7..6ad3c44 100644 --- a/src/matteloop/ui/main_window.py +++ b/src/matteloop/ui/main_window.py @@ -223,6 +223,15 @@ def _build(self) -> None: ) for widget in (self.timeline_widget, self.original_canvas, self.inspector): widget.command_requested.connect(self._services.dispatch) + self.inspector.exclusion_controls.edit_toggled.connect( + self.original_canvas.set_exclusion_edit + ) + self.original_canvas.exclusion_selection_changed.connect( + self.inspector.exclusion_controls.set_selected_exclusion + ) + self.original_canvas.exclusion_edit_toggled.connect( + self.inspector.exclusion_controls.edit_button.setChecked + ) self.render_button.clicked.connect( lambda: self._services.dispatch(RenderVideoRequested()) ) diff --git a/src/matteloop/ui/parameter_presentation.py b/src/matteloop/ui/parameter_presentation.py index 1d8d47d..e3ea84c 100644 --- a/src/matteloop/ui/parameter_presentation.py +++ b/src/matteloop/ui/parameter_presentation.py @@ -11,7 +11,7 @@ output_directory_for_source, output_filename_for_source, ) -from matteloop.core.specs import EdgeMode, TransformSpec +from matteloop.core.specs import CropSpec, EdgeMode, TransformSpec from matteloop.core.state import AppState, ArtifactResult @@ -25,6 +25,8 @@ class ParameterPresentation: alpha_threshold: Decimal padding: int stretch_x: Decimal + exclusions: tuple[CropSpec, ...] + exclusions_before_model: bool output_directory: Path | None output_filename: str max_mib: Decimal @@ -62,6 +64,8 @@ def present_parameters(state: AppState) -> ParameterPresentation: alpha_threshold=parameters.alpha_threshold, padding=parameters.padding, stretch_x=parameters.stretch_x, + exclusions=parameters.exclusions, + exclusions_before_model=parameters.exclusions_before_model, output_directory=directory, output_filename=filename, max_mib=parameters.max_mib, diff --git a/src/matteloop/ui/preferences.py b/src/matteloop/ui/preferences.py index d1c5493..310c752 100644 --- a/src/matteloop/ui/preferences.py +++ b/src/matteloop/ui/preferences.py @@ -20,6 +20,8 @@ "alpha_threshold", "padding", "stretch_x", + "exclusions", + "exclusions_before_model", "output_directory", "output_filename", "max_mib", @@ -55,6 +57,16 @@ def persist_parameters(settings: QSettings, parameters: ParameterState) -> None: ) settings.setValue(f"{_PREFIX}padding", parameters.padding) settings.setValue(f"{_PREFIX}stretch_x", str(parameters.stretch_x)) + settings.setValue( + f"{_PREFIX}exclusions", + ";".join( + ",".join(str(value) for value in (e.x, e.y, e.width, e.height)) + for e in parameters.exclusions + ), + ) + settings.setValue( + f"{_PREFIX}exclusions_before_model", parameters.exclusions_before_model + ) if parameters.output_directory is None: settings.remove(f"{_PREFIX}output_directory") else: diff --git a/src/matteloop/ui/preview_canvas.py b/src/matteloop/ui/preview_canvas.py index 3742988..fc368fd 100644 --- a/src/matteloop/ui/preview_canvas.py +++ b/src/matteloop/ui/preview_canvas.py @@ -274,14 +274,14 @@ def __init__( self, parent: QWidget | None = None, *, runtime_root: Path | None = None ) -> None: super().__init__(parent) - from matteloop.ui.crop_canvas import CropCanvas + from matteloop.ui.exclusion_canvas import ExclusionCanvas from matteloop.ui.result_player import ResultPlayerCanvas self.setObjectName("preview_stage") layout = QHBoxLayout(self) layout.setContentsMargins(16, 16, 16, 16) layout.setSpacing(8) - self.original_canvas = CropCanvas( + self.original_canvas = ExclusionCanvas( title=QCoreApplication.translate("PreviewCanvas", "Original") ) self.result_canvas = ResultPlayerCanvas(runtime_root=runtime_root) diff --git a/src/matteloop/ui/request_builder.py b/src/matteloop/ui/request_builder.py index 1ad5ee2..38831b3 100644 --- a/src/matteloop/ui/request_builder.py +++ b/src/matteloop/ui/request_builder.py @@ -95,12 +95,18 @@ def _render_request( model_id=parameters.model_id, edge_mode=parameters.edge_mode, execution_provider=parameters.execution_provider, + exclusions=( + parameters.exclusions + if parameters.exclusions_before_model + else () + ), ), framing=FramingSpec( trim=parameters.trim, alpha_threshold=parameters.alpha_threshold, padding=parameters.padding, stretch_x=parameters.stretch_x, + exclusions=parameters.exclusions, ), output=OutputSpec.from_mib( output_directory_for_source(parameters, inputs.source), diff --git a/src/matteloop/ui/result_player.py b/src/matteloop/ui/result_player.py index 704dba1..55d0032 100644 --- a/src/matteloop/ui/result_player.py +++ b/src/matteloop/ui/result_player.py @@ -31,7 +31,13 @@ from PySide6.QtWidgets import QPushButton, QWidget from matteloop.core.crop import fit_crop_aspect -from matteloop.core.geometry import _MEDIA_INSET, FramingPlan, apply_framing +from matteloop.core.exclusion import exclude_alpha +from matteloop.core.geometry import ( + _MEDIA_INSET, + FramingPlan, + PixelBounds, + apply_framing, +) from matteloop.core.parameters import TransformChanged from matteloop.core.specs import CropSpec, TransformSpec from matteloop.core.transform import apply_transform @@ -393,6 +399,7 @@ def __init__( *, budget: int = PLAYER_CACHE_BUDGET_BYTES, cancelled: Callable[[], bool] = _no_cancellation, + exclusions: tuple[PixelBounds, ...] = (), ) -> None: super().__init__() self._workspace = workspace @@ -404,6 +411,7 @@ def __init__( self._generation = generation self._budget = budget self._cancelled = cancelled + self._exclusions = exclusions @Slot() def run(self) -> None: @@ -417,6 +425,7 @@ def run(self) -> None: self._delays, self._budget, self._cancelled, + exclusions=self._exclusions, ) except _FrameLoadCancelled: pass @@ -442,6 +451,8 @@ def _load_player_frames( delays: tuple[int, ...], budget: int, cancelled: Callable[[], bool] = _no_cancellation, + *, + exclusions: tuple[PixelBounds, ...] = (), ) -> PlayerFrames: frame_count = manifest.frame_count display_size, cached = _fit_budget(frame_count, plan.output_size, budget) @@ -464,6 +475,7 @@ def _load_player_frames( plan, transform, display_size, + exclusions, ) framed_images.append(framed_qimage) transformed_images.append(transformed_qimage) @@ -484,9 +496,11 @@ def _load_one_frame( plan: FramingPlan, transform: TransformSpec, display_size: tuple[int, int], + exclusions: tuple[PixelBounds, ...] = (), ) -> tuple[QImage, QImage]: cut = frame_reader.read(workspace, frame) try: + exclude_alpha(cut, exclusions) framed = apply_framing(cut, plan) finally: cut.close() diff --git a/src/matteloop/ui/transform_stage.py b/src/matteloop/ui/transform_stage.py index f8bb700..f35f2ee 100644 --- a/src/matteloop/ui/transform_stage.py +++ b/src/matteloop/ui/transform_stage.py @@ -17,7 +17,6 @@ import threading from collections.abc import Callable, Iterator, Mapping from dataclasses import dataclass, replace -from decimal import Decimal from itertools import count from typing import TYPE_CHECKING, Protocol, cast @@ -26,12 +25,17 @@ from PySide6.QtWidgets import QMessageBox, QWidget from matteloop.core.crop import clamp_crop -from matteloop.core.geometry import FramingPlan, PixelBounds, union_alpha_bounds +from matteloop.core.exclusion import ( + cut_exclusions, + exclude_alpha, + union_alpha_bounds_or_none, +) +from matteloop.core.geometry import FramingPlan, PixelBounds from matteloop.core.parameters import ParameterState, TransformChanged from matteloop.core.specs import CropSpec, FramingSpec, TransformSpec from matteloop.core.state import AppState from matteloop.core.timebase import webp_delays -from matteloop.jobs.transform_stage import framing_plan +from matteloop.jobs.transform_stage import cached_union, framing_plan from matteloop.jobs.transform_store import load_transform from matteloop.jobs.workspace import ( CutFrame, @@ -351,7 +355,9 @@ def _state_changed(self, state: AppState) -> None: return parameters = state.parameters framing = _framing_from_parameters(parameters) - framing_changed = framing != self._framing + framing_changed = _effective_framing_changed( + framing, self._framing, self._session + ) edit_scan_completed = ( self._edited_cuts_request_id is not None and state.edited_cuts_request_id is None @@ -361,8 +367,7 @@ def _state_changed(self, state: AppState) -> None: refresh_external_edits = edit_scan_completed or edited_cuts_became_true self._edited_cuts = state.edited_cuts self._edited_cuts_request_id = state.edited_cuts_request_id - if framing_changed: - self._framing = framing + self._framing = framing if self._session is not None and (framing_changed or refresh_external_edits): self._schedule_facts(refresh_external_edits=refresh_external_edits) transform = parameters.transform @@ -564,6 +569,7 @@ def _start_frame_load(self) -> None: cancel_event = threading.Event() self._frame_cancel_event = cancel_event transform = self._store.state.parameters.transform + crop = _crop_from_manifest(session.manifest) worker = FrameLoadWorker( session.workspace, session.manifest, @@ -574,6 +580,7 @@ def _start_frame_load(self) -> None: generation, budget=self._cache_budget, cancelled=cancel_event.is_set, + exclusions=cut_exclusions(self._framing.exclusions, crop), ) thread = QThread(self) worker.moveToThread(thread) @@ -644,6 +651,28 @@ def _framing_from_parameters(parameters: ParameterState) -> FramingSpec: parameters.alpha_threshold, parameters.padding, parameters.stretch_x, + parameters.exclusions, + ) + + +def _effective_framing_changed( + current: FramingSpec, + previous: FramingSpec, + session: CutSession | None, +) -> bool: + """Compare framing inputs after source regions are mapped into cut space.""" + if ( + current.trim != previous.trim + or current.alpha_threshold != previous.alpha_threshold + or current.padding != previous.padding + or current.stretch_x != previous.stretch_x + ): + return True + if session is None: + return current.exclusions != previous.exclusions + crop = _crop_from_manifest(session.manifest) + return cut_exclusions(current.exclusions, crop) != cut_exclusions( + previous.exclusions, crop ) @@ -667,6 +696,24 @@ def _fps_from_manifest(manifest: CutManifest) -> int: return fps +def _crop_from_manifest(manifest: CutManifest) -> CropSpec: + raw_crop = manifest.cache_key_inputs["crop"] + if not isinstance(raw_crop, Mapping): + raise ValueError("manifest crop inputs are not a mapping") + x = raw_crop["x"] + y = raw_crop["y"] + width = raw_crop["width"] + height = raw_crop["height"] + if ( + type(x) is not int + or type(y) is not int + or type(width) is not int + or type(height) is not int + ): + raise ValueError("manifest crop inputs are not integers") + return CropSpec(x, y, width, height) + + def _compute_facts( session: CutSession, framing: FramingSpec, frame_reader: FrameReader, cancelled: Callable[[], bool] = lambda: False, @@ -690,32 +737,36 @@ def _compute_facts( def _resolve_union( session: CutSession, framing: FramingSpec, frame_reader: FrameReader, cancelled: Callable[[], bool] = lambda: False, -) -> PixelBounds: +) -> PixelBounds | None: + crop = _crop_from_manifest(session.manifest) metadata = ( None if session.manifest.edited else session.manifest.union_metadata ) - if metadata is not None: - try: - matches = Decimal(metadata.alpha_threshold) == framing.alpha_threshold - except ArithmeticError: - matches = False - if matches: - left, top, right, bottom = metadata.bounds - return PixelBounds(left, top, right, bottom) - return union_alpha_bounds( - _iter_stored_frames(session, frame_reader, cancelled), framing.alpha_threshold + cached = cached_union(metadata, framing, crop, session.manifest.cache_key) + if cached is not None: + return cached + return union_alpha_bounds_or_none( + _iter_stored_frames( + session, + frame_reader, + cancelled, + exclusions=cut_exclusions(framing.exclusions, crop), + ), + framing.alpha_threshold, ) def _iter_stored_frames( session: CutSession, frame_reader: FrameReader, cancelled: Callable[[], bool] = lambda: False, + *, exclusions: tuple[PixelBounds, ...] = (), ) -> Iterator[Image.Image]: for frame in session.manifest.frames: if cancelled(): raise _CutFactsCancelled image = frame_reader.read(session.workspace, frame) try: + exclude_alpha(image, exclusions) yield image finally: image.close() diff --git a/src/matteloop/ui/workspace_presentation.py b/src/matteloop/ui/workspace_presentation.py index 5f4bddd..410016c 100644 --- a/src/matteloop/ui/workspace_presentation.py +++ b/src/matteloop/ui/workspace_presentation.py @@ -147,6 +147,25 @@ def request_for_workspace(manifest: CutManifest, base: RenderRequest) -> RenderR model = _mapping(inputs["model"]) edge_settings = _mapping(inputs["edge_settings"]) matting = _mapping(edge_settings["alpha_matting"]) + model_exclusions = inputs.get("model_exclusions") + segmentation_exclusions: tuple[CropSpec, ...] = () + if model_exclusions is not None: + model_box_values = _mapping(model_exclusions) + raw_boxes = model_box_values["boxes"] + if not isinstance(raw_boxes, (list, tuple)): + raise ValueError("manifest model exclusions are not an array") + crop_x = _integer(crop["x"]) + crop_y = _integer(crop["y"]) + segmentation_exclusions = tuple( + CropSpec( + crop_x + _integer(box[0]), + crop_y + _integer(box[1]), + _integer(box[2]) - _integer(box[0]), + _integer(box[3]) - _integer(box[1]), + ) + for box in raw_boxes + if isinstance(box, (list, tuple)) and len(box) == 4 + ) mode = { "standard": EdgeMode.STANDARD, "decontaminate": EdgeMode.DECONTAMINATE_COLORS, @@ -173,6 +192,7 @@ def request_for_workspace(manifest: CutManifest, base: RenderRequest) -> RenderR _integer(matting["erode_size"]), ), base.segmentation.execution_provider, + exclusions=segmentation_exclusions, ), framing=base.framing, output=base.output, diff --git a/tests/core/test_crop.py b/tests/core/test_crop.py index 7851fe8..7730870 100644 --- a/tests/core/test_crop.py +++ b/tests/core/test_crop.py @@ -2,7 +2,10 @@ from fractions import Fraction +import pytest + from matteloop.core.crop import ( + _orientation_transform, centered_crop_for_aspect, clamp_crop, crop_from_drag, @@ -13,12 +16,14 @@ ) from matteloop.core.geometry import ( CropGeometryState, + MediaTransform, PointF, RectF, SizeF, build_crop_geometry, ) from matteloop.core.specs import CropSpec +from matteloop.jobs.source import _display_dimensions def test_dragging_crop_moves_and_clamps_in_oriented_source_pixels() -> None: @@ -84,6 +89,90 @@ def test_oriented_crop_geometry_maps_rotation_and_pixel_aspect_once() -> None: assert oriented_point_from_widget(geometry, PointF(0, 0)) == PointF(0, 0) +@pytest.mark.parametrize("rotation", [0, 90, 180, 270]) +def test_oriented_crop_mapping_uses_rounded_decoder_extent(rotation: int) -> None: + """720x480 at pixel aspect 32/27 (NTSC DVD 16:9) decodes to 853, not 853.33. + + Asserting against ``_display_dimensions`` rather than a written-out 853 + is the point: the canvas and the decoder each round the same product, and + this is the assertion that fails if either one drifts from the other. + """ + decoded = _display_dimensions(720, 480, Fraction(32, 27), rotation) + + transform = _orientation_transform(720, 480, rotation, 32 / 27) + + assert transform.viewport == SizeF(*decoded) + + +@pytest.mark.parametrize("rotation", [0, 90, 180, 270]) +@pytest.mark.parametrize("pixel_aspect", [1.0, 2.0, 1.5, 0.75]) +def test_oriented_crop_mapping_round_trips_inside_source_frame( + rotation: int, pixel_aspect: float +) -> None: + source_size = SizeF(16, 8) + crop = CropSpec(4, 0, 4, 8) + display_size = MediaTransform( + source_size=source_size, + viewport=SizeF(1, 1), + rotation=rotation, + pixel_aspect=pixel_aspect, + ).oriented_display_size + + mapped = oriented_rect_to_source_rect( + crop, + source_width=16, + source_height=8, + rotation=rotation, + pixel_aspect=pixel_aspect, + ) + + assert mapped.x >= 0 + assert mapped.y >= 0 + assert mapped.right <= source_size.width + assert mapped.bottom <= source_size.height + + restored = MediaTransform( + source_size=source_size, + viewport=display_size, + rotation=rotation, + pixel_aspect=pixel_aspect, + ).source_rect_to_widget(mapped) + + assert restored.x == pytest.approx(crop.x) + assert restored.y == pytest.approx(crop.y) + assert restored.width == pytest.approx(crop.width) + assert restored.height == pytest.approx(crop.height) + + +@pytest.mark.parametrize("rotation", [0, 90, 180, 270]) +@pytest.mark.parametrize("pixel_aspect", [1.0, 2.0, 1.5, 0.75]) +def test_oriented_point_mapping_round_trips_through_display_geometry( + rotation: int, pixel_aspect: float +) -> None: + source_size = SizeF(16, 8) + display_size = MediaTransform( + source_size=source_size, + viewport=SizeF(1, 1), + rotation=rotation, + pixel_aspect=pixel_aspect, + ).oriented_display_size + geometry = build_crop_geometry( + state=CropGeometryState( + source_size=source_size, + crop=RectF(0, 0, 8, 8), + rotation=rotation, + pixel_aspect=pixel_aspect, + ), + viewport=display_size, + dpr=1, + ) + + restored = oriented_point_from_widget(geometry, PointF(2, 3)) + + assert restored.x == pytest.approx(2) + assert restored.y == pytest.approx(3) + + def test_fit_crop_aspect_on_a_corner_keeps_the_opposite_corner_fixed() -> None: fitted = fit_crop_aspect( CropSpec(2, 3, 6, 5), diff --git a/tests/core/test_exclusion.py b/tests/core/test_exclusion.py new file mode 100644 index 0000000..73b0176 --- /dev/null +++ b/tests/core/test_exclusion.py @@ -0,0 +1,113 @@ +from __future__ import annotations + +from decimal import Decimal + +import pytest +from PIL import Image + +from matteloop.core.errors import ErrorCode, ValidationError +from matteloop.core.exclusion import ( + cut_exclusions, + exclude_alpha, + union_alpha_bounds_or_none, +) +from matteloop.core.geometry import PixelBounds +from matteloop.core.specs import CropSpec + + +def test_cut_exclusions_translates_by_the_crop_origin_and_clips_to_the_crop() -> None: + crop = CropSpec(10, 20, 100, 80) + exclusions = ( + CropSpec(0, 10, 30, 30), + CropSpec(80, 60, 60, 60), + ) + + assert cut_exclusions(exclusions, crop) == ( + PixelBounds(0, 0, 20, 20), + PixelBounds(70, 40, 100, 80), + ) + + +def test_cut_exclusions_drops_a_region_outside_the_crop() -> None: + assert cut_exclusions((CropSpec(0, 0, 5, 5),), CropSpec(10, 10, 20, 20)) == () + + +def test_cut_exclusions_is_order_independent_and_deduplicated() -> None: + crop = CropSpec(10, 10, 100, 100) + first = CropSpec(40, 50, 20, 30) + second = CropSpec(20, 20, 10, 10) + + assert cut_exclusions((first, second, first), crop) == cut_exclusions( + (second, first), crop + ) + + +def test_blank_model_input_fills_each_box_and_nothing_else() -> None: + from matteloop.core.exclusion import blank_model_input + + image = Image.new("RGBA", (8, 6)) + image.putdata( + tuple((x, y, x + y, 20 + x * 8 + y) for y in range(6) for x in range(8)) + ) + original = image.copy() + boxes = (PixelBounds(1, 1, 3, 3), PixelBounds(5, 0, 7, 2)) + + blank_model_input(image, boxes) + + for y in range(6): + for x in range(8): + inside = (1 <= x < 3 and 1 <= y < 3) or (5 <= x < 7 and y < 2) + expected = ( + (124, 116, 104, original.getpixel((x, y))[3]) + if inside + else original.getpixel((x, y)) + ) + assert image.getpixel((x, y)) == expected + + +def test_blank_model_input_with_no_boxes_leaves_the_image_byte_identical() -> None: + from matteloop.core.exclusion import blank_model_input + + image = Image.new("RGBA", (4, 3), (10, 20, 30, 40)) + before = image.tobytes() + + blank_model_input(image, ()) + + assert image.tobytes() == before + + +def test_exclude_alpha_zeroes_rgba_inside_each_box_and_nothing_else() -> None: + image = Image.new("RGBA", (6, 6), (10, 20, 30, 255)) + boxes = (PixelBounds(1, 1, 3, 3), PixelBounds(4, 0, 6, 2)) + + exclude_alpha(image, boxes) + + for y in range(6): + for x in range(6): + excluded = (1 <= x < 3 and 1 <= y < 3) or (4 <= x < 6 and y < 2) + assert image.getpixel((x, y)) == ( + (0, 0, 0, 0) if excluded else (10, 20, 30, 255) + ) + + +def test_union_alpha_bounds_or_none_returns_none_for_an_empty_mask() -> None: + image = Image.new("RGBA", (8, 8), (20, 30, 40, 0)) + + assert union_alpha_bounds_or_none((image,), Decimal("2")) is None + + +@pytest.mark.parametrize( + "frames", + [ + (Image.new("RGBA", (8, 8)), Image.new("RGBA", (9, 8))), + (Image.new("RGB", (8, 8)),), + ], + ids=["mismatched-canvases", "non-rgba-frame"], +) +def test_union_alpha_bounds_or_none_propagates_invalid_frame_errors( + frames: tuple[Image.Image, ...], +) -> None: + with pytest.raises(ValidationError) as exc: + union_alpha_bounds_or_none(frames, Decimal("2")) + + assert exc.value.code is ErrorCode.INVALID_FRAMING diff --git a/tests/core/test_fingerprints.py b/tests/core/test_fingerprints.py index 5c4ebdf..b348781 100644 --- a/tests/core/test_fingerprints.py +++ b/tests/core/test_fingerprints.py @@ -4,6 +4,7 @@ import json import os import threading +from collections.abc import Mapping from dataclasses import replace from decimal import Decimal, localcontext from fractions import Fraction @@ -18,6 +19,7 @@ complete_source_sha256, cut_cache_key, cut_cache_key_inputs, + cut_union_fingerprint, provisional_source_fingerprint, render_fingerprint, union_fingerprint, @@ -411,6 +413,82 @@ def test_union_identity_tracks_only_cut_content_and_alpha_threshold( assert union_fingerprint(new_threshold, cut_key=cut_key) != baseline +def test_render_and_preview_identity_track_effective_exclusions_only( + tmp_path: Path, +) -> None: + request = request_a(tmp_path) + outside = replace( + request, + framing=replace( + request.framing, exclusions=(CropSpec(0, 0, 1, 1),) + ), + ) + inside = replace( + request, + framing=replace( + request.framing, exclusions=(CropSpec(10, 10, 20, 20),) + ), + ) + + assert render_fingerprint(outside, cut_key="ef" * 32) == render_fingerprint( + request, cut_key="ef" * 32 + ) + assert preview_fingerprint(outside, Fraction(1, 5)) == preview_fingerprint( + request, Fraction(1, 5) + ) + assert render_fingerprint(inside, cut_key="ef" * 32) != render_fingerprint( + request, cut_key="ef" * 32 + ) + assert preview_fingerprint(inside, Fraction(1, 5)) != preview_fingerprint( + request, Fraction(1, 5) + ) + + +def test_union_identity_tracks_effective_exclusions(tmp_path: Path) -> None: + request = request_a(tmp_path) + first_region = CropSpec(10, 10, 20, 20) + second_region = CropSpec(40, 40, 10, 10) + first = replace( + request, + framing=replace( + request.framing, exclusions=(first_region, second_region) + ), + ) + reordered = replace( + request, + framing=replace( + request.framing, + exclusions=(second_region, first_region), + ), + ) + outside = replace( + request, + framing=replace( + request.framing, exclusions=(CropSpec(0, 0, 1, 1),) + ), + ) + + assert union_fingerprint(first, cut_key="ef" * 32) == union_fingerprint( + reordered, cut_key="ef" * 32 + ) + assert union_fingerprint(first, cut_key="ef" * 32) != union_fingerprint( + outside, cut_key="ef" * 32 + ) + + +def test_identities_without_exclusions_match_the_previous_schema( + tmp_path: Path, +) -> None: + request = replace(request_a(tmp_path), framing=FramingSpec()) + + assert "exclusions" not in fingerprints_module._framing(request) + assert cut_union_fingerprint( + request.framing, request.crop, cut_key="ef" * 32 + ) == ( + "8e9f72c32c74a7feb4a7bc60345c74dfeee018fd73d4864d6bbc92e5bc96fb17" + ) + + def test_render_fingerprint_distinguishes_long_exact_decimal_values( tmp_path: Path, ) -> None: @@ -530,7 +608,9 @@ def test_fingerprints_reject_malformed_canonical_identities( assert exc.value.code is ErrorCode.INVALID_RENDER_REQUEST -def test_cut_key_uses_a_canonical_versioned_json_schema(tmp_path: Path) -> None: +def test_cut_key_without_model_exclusions_matches_the_canonical_schema( + tmp_path: Path, +) -> None: request = request_a(tmp_path) canonical_json = json.dumps( { @@ -567,6 +647,143 @@ def test_cut_key_uses_a_canonical_versioned_json_schema(tmp_path: Path) -> None: cut_cache_key(request, source_sha256=SOURCE_SHA, model_weight_sha256=MODEL_SHA) == hashlib.sha256(canonical_json.encode("utf-8")).hexdigest() ) + for segmentation in ( + request.segmentation, + replace(request.segmentation, exclusions=()), + ): + no_regions = replace(request, segmentation=segmentation) + inputs = cut_cache_key_inputs( + no_regions, + source_sha256=SOURCE_SHA, + model_weight_sha256=MODEL_SHA, + ) + assert "model_exclusions" not in inputs + assert cut_cache_key( + no_regions, + source_sha256=SOURCE_SHA, + model_weight_sha256=MODEL_SHA, + ) == cut_cache_key( + request, + source_sha256=SOURCE_SHA, + model_weight_sha256=MODEL_SHA, + ) + + +def test_cut_key_ignores_a_model_exclusion_outside_the_crop_or_clipped_to_nothing( + tmp_path: Path, +) -> None: + request = request_a(tmp_path) + baseline = cut_cache_key( + request, source_sha256=SOURCE_SHA, model_weight_sha256=MODEL_SHA + ) + for exclusion in (CropSpec(0, 0, 1, 1), CropSpec(0, 3, 2, 1)): + changed = replace( + request, + segmentation=replace( + request.segmentation, exclusions=(exclusion,) + ), + ) + inputs = cut_cache_key_inputs( + changed, + source_sha256=SOURCE_SHA, + model_weight_sha256=MODEL_SHA, + ) + assert "model_exclusions" not in inputs + assert cut_cache_key( + changed, source_sha256=SOURCE_SHA, model_weight_sha256=MODEL_SHA + ) == baseline + + +def test_cut_key_tracks_model_exclusion_boxes_and_the_fill_identifier( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + request = request_a(tmp_path) + first = replace( + request, + segmentation=replace( + request.segmentation, + exclusions=(CropSpec(2, 3, 20, 30), CropSpec(40, 50, 10, 12)), + ), + ) + second = replace( + first, + segmentation=replace( + first.segmentation, + exclusions=(CropSpec(2, 3, 20, 31), CropSpec(40, 50, 10, 12)), + ), + ) + + first_inputs = cut_cache_key_inputs( + first, source_sha256=SOURCE_SHA, model_weight_sha256=MODEL_SHA + ) + assert first_inputs["model_exclusions"] == { + "fill": fingerprints_module.MODEL_FILL_ID, + "boxes": ((0, 0, 20, 30), (38, 47, 48, 59)), + } + original_key = cut_cache_key( + first, source_sha256=SOURCE_SHA, model_weight_sha256=MODEL_SHA + ) + assert original_key != cut_cache_key( + second, source_sha256=SOURCE_SHA, model_weight_sha256=MODEL_SHA + ) + + monkeypatch.setattr(fingerprints_module, "MODEL_FILL_ID", "future-fill") + assert original_key != cut_cache_key( + first, source_sha256=SOURCE_SHA, model_weight_sha256=MODEL_SHA + ) + + +def test_cut_key_inputs_with_model_exclusions_are_recursively_immutable( + tmp_path: Path, +) -> None: + request = replace( + request_a(tmp_path), + segmentation=SegmentationSpec( + exclusions=(CropSpec(2, 3, 20, 30),) + ), + ) + + inputs = cut_cache_key_inputs( + request, source_sha256=SOURCE_SHA, model_weight_sha256=MODEL_SHA + ) + model_exclusions = inputs["model_exclusions"] + assert isinstance(model_exclusions, Mapping) + assert isinstance(model_exclusions["boxes"], tuple) + assert isinstance(model_exclusions["boxes"][0], tuple) + with pytest.raises(TypeError): + model_exclusions["boxes"] = () # type: ignore[index] + + +def test_preview_identity_tracks_model_exclusions(tmp_path: Path) -> None: + request = request_a(tmp_path) + changed = replace( + request, + segmentation=replace( + request.segmentation, exclusions=(CropSpec(2, 3, 20, 30),) + ), + ) + + assert preview_fingerprint(changed, Fraction(1, 5)) != preview_fingerprint( + request, Fraction(1, 5) + ) + + +def test_render_and_union_identities_ignore_model_exclusions(tmp_path: Path) -> None: + request = request_a(tmp_path) + changed = replace( + request, + segmentation=replace( + request.segmentation, exclusions=(CropSpec(2, 3, 20, 30),) + ), + ) + cut_key = "ef" * 32 + + assert render_fingerprint(changed, cut_key=cut_key) == render_fingerprint( + request, cut_key=cut_key + ) + assert union_fingerprint(changed, cut_key=cut_key) == union_fingerprint( + request, cut_key=cut_key + ) def test_preview_identity_tracks_playhead_and_matting_values(tmp_path: Path) -> None: diff --git a/tests/core/test_parameters.py b/tests/core/test_parameters.py index b6fd827..bc89d87 100644 --- a/tests/core/test_parameters.py +++ b/tests/core/test_parameters.py @@ -11,6 +11,8 @@ V1_MODEL_IDS, AlphaThresholdChanged, EdgeModeChanged, + ExclusionsBeforeModelChanged, + ExclusionsChanged, ExecutionProviderChanged, GlobalTrimChanged, ModelChanged, @@ -73,6 +75,7 @@ def test_parameter_defaults_match_the_v1_mapping() -> None: assert parameters.alpha_threshold == Decimal("2.0") assert parameters.padding == 0 assert parameters.stretch_x == Decimal("1.0") + assert parameters.exclusions == () assert parameters.max_mib == Decimal("0") @@ -117,6 +120,73 @@ def test_segmentation_parameter_changes_stale_the_current_preview() -> None: assert changed_again.preview is PreviewState.STALE +def test_exclusions_before_model_invalidates_the_preview_as_segmentation() -> None: + state = replace( + _current(), + parameters=replace( + _current().parameters, + exclusions=(CropSpec(10, 10, 20, 20),), + ), + ) + + changed = reduce(state, ExclusionsBeforeModelChanged(True)) + + assert changed.parameters.exclusions_before_model is True + assert changed.preview is PreviewState.STALE + assert changed.stale_category is PreviewInvalidationReason.SEGMENTATION + + +def test_exclusions_before_model_with_no_effective_region_does_not_invalidate() -> None: + state = replace( + _current(), + parameters=replace( + _current().parameters, + exclusions=(CropSpec(200, 200, 20, 20),), + ), + ) + + changed = reduce(state, ExclusionsBeforeModelChanged(True)) + + assert changed.parameters.exclusions_before_model is True + assert changed.preview is PreviewState.CURRENT + assert changed.stale_category is None + + +def test_region_change_invalidates_as_segmentation_while_the_switch_is_on() -> None: + initial = CropSpec(10, 10, 20, 20) + state = replace( + _current(), + parameters=replace( + _current().parameters, + exclusions=(initial,), + exclusions_before_model=True, + ), + ) + + changed = reduce( + state, ExclusionsChanged((CropSpec(20, 20, 20, 20),)) + ) + + assert changed.preview is PreviewState.STALE + assert changed.stale_category is PreviewInvalidationReason.SEGMENTATION + + +def test_parameters_reset_clears_exclusions_before_model() -> None: + state = replace( + _current(), + parameters=replace( + _current().parameters, + exclusions=(CropSpec(10, 10, 20, 20),), + exclusions_before_model=True, + ), + ) + + reset = reduce(state, ParametersReset()) + + assert reset.parameters.exclusions_before_model is False + assert reset.stale_category is PreviewInvalidationReason.SEGMENTATION + + def test_execution_provider_changes_stale_the_current_preview() -> None: changed = reduce(_current(), ExecutionProviderChanged("CUDAExecutionProvider")) @@ -154,6 +224,33 @@ def test_cleanup_parameter_changes_stale_the_current_preview() -> None: assert state.stale_category is PreviewInvalidationReason.CROP_CLEANUP +def test_exclusions_changed_invalidates_the_preview_as_crop_cleanup() -> None: + region = CropSpec(8, 8, 16, 16) + + changed = reduce(_current(), ExclusionsChanged((region,))) + + assert changed.parameters.exclusions == (region,) + assert changed.preview is PreviewState.STALE + assert changed.stale_category is PreviewInvalidationReason.CROP_CLEANUP + + +def test_moving_a_region_outside_the_crop_does_not_invalidate_the_preview() -> None: + state = replace( + _current(), + crop=CropSpec(32, 32, 32, 32), + parameters=replace( + _current().parameters, exclusions=(CropSpec(0, 0, 8, 8),) + ), + ) + + changed = reduce(state, ExclusionsChanged((CropSpec(1, 1, 8, 8),))) + + assert changed is not state + assert changed.parameters.exclusions == (CropSpec(1, 1, 8, 8),) + assert changed.preview is state.preview + assert changed.stale_category is state.stale_category + + def test_output_fps_changes_sampling_and_stales_the_current_preview() -> None: changed = reduce(_current(), OutputFpsChanged(90)) @@ -176,6 +273,7 @@ def test_reset_restores_inspector_parameters_and_preserves_other_state() -> None alpha_threshold=Decimal("0.4"), padding=1, stretch_x=Decimal("1.2"), + exclusions=(CropSpec(4, 5, 20, 20),), output_directory=Path("exports"), output_filename="chosen.webp", max_mib=Decimal("12.5"), @@ -207,6 +305,7 @@ def test_reset_restores_inspector_parameters_and_preserves_other_state() -> None "alpha_threshold", "padding", "stretch_x", + "exclusions", "max_mib", ): assert getattr(reset.parameters, name) == getattr(defaults, name) @@ -386,3 +485,24 @@ def test_unrelated_invalid_saved_value_does_not_reset_valid_preferences() -> Non assert parameters.fps == 60 assert parameters.alpha_threshold == Decimal("2.0") assert parameters.padding == 12 + + +def test_exclusions_settings_string_round_trips_and_malformed_entries_fall_back_to_none( +) -> None: + exclusions = (CropSpec(1, 2, 30, 40), CropSpec(50, 60, 7, 8)) + + assert parameters_from_values( + {"exclusions": "1,2,30,40;50,60,7,8"} + ).exclusions == exclusions + assert parameters_from_values({"exclusions": "1,2,30"}).exclusions == () + assert parameters_from_values({"exclusions": "1,2,3,4;bad"}).exclusions == () + + +def test_exclusions_before_model_settings_round_trip_and_malformed_value_fall_back( +) -> None: + assert parameters_from_values( + {"exclusions_before_model": "true"} + ).exclusions_before_model is True + assert parameters_from_values( + {"exclusions_before_model": "not-a-bool"} + ).exclusions_before_model is False diff --git a/tests/core/test_specs.py b/tests/core/test_specs.py index 805704d..bfe9618 100644 --- a/tests/core/test_specs.py +++ b/tests/core/test_specs.py @@ -136,6 +136,40 @@ def test_crop_accepts_right_and_bottom_source_edges() -> None: crop.validate_for(100, 100) +@pytest.mark.parametrize( + "exclusions", + [ + [CropSpec(0, 0, 10, 10)], + (CropSpec(0, 0, 10, 10), (0, 0, 10, 10)), + ], + ids=["not-a-tuple", "contains-a-non-crop"], +) +def test_framing_spec_rejects_exclusions_that_are_not_crop_specs( + exclusions: object, +) -> None: + with pytest.raises(ValidationError) as exc: + FramingSpec(exclusions=exclusions) # type: ignore[arg-type] + + assert exc.value.code is ErrorCode.INVALID_FRAMING + + +@pytest.mark.parametrize( + "exclusions", + [ + [CropSpec(0, 0, 10, 10)], + (CropSpec(0, 0, 10, 10), (0, 0, 10, 10)), + ], + ids=["not-a-tuple", "contains-a-non-crop"], +) +def test_segmentation_spec_rejects_exclusions_that_are_not_crop_specs( + exclusions: object, +) -> None: + with pytest.raises(ValidationError) as exc: + SegmentationSpec(exclusions=exclusions) # type: ignore[arg-type] + + assert exc.value.code is ErrorCode.INVALID_SEGMENTATION + + @pytest.mark.parametrize( ("crop", "source_width", "source_height"), [ diff --git a/tests/core/test_state.py b/tests/core/test_state.py index 4d7314a..6ddadb0 100644 --- a/tests/core/test_state.py +++ b/tests/core/test_state.py @@ -1,9 +1,12 @@ from __future__ import annotations -from dataclasses import FrozenInstanceError, dataclass +from dataclasses import FrozenInstanceError, dataclass, replace +from fractions import Fraction import pytest +from matteloop.core.exclusion import cut_exclusions +from matteloop.core.geometry import PixelBounds from matteloop.core.parameters import TransformChanged from matteloop.core.specs import CropSpec, TransformSpec from matteloop.core.state import ( @@ -47,6 +50,14 @@ SOURCE_VALUE = "metadata" +@dataclass(frozen=True) +class SizedSource: + width: int + height: int + duration: Fraction = Fraction(2) + average_rate: Fraction = Fraction(30) + + def loading_state( *, source_id: str = SOURCE_ID, request_id: str = SOURCE_REQUEST_ID ) -> AppState: @@ -881,3 +892,30 @@ def test_loading_a_new_source_resets_the_transform_but_keeps_other_parameters() assert loading_next_source.parameters.model_id == with_transform.parameters.model_id assert loading_next_source.parameters.trim == with_transform.parameters.trim assert loading_next_source.parameters.padding == with_transform.parameters.padding + + +def test_loading_a_smaller_source_clips_carried_regions_and_drops_those_outside( +) -> None: + carried = replace( + ready_state(), + parameters=replace( + ready_state().parameters, + exclusions=( + CropSpec(60, 70, 80, 20), + CropSpec(120, 0, 10, 10), + ), + ), + ) + loading = reduce( + carried, + SourceLoadRequested(source_id="smaller", request_id="load-2"), + ) + loaded = reduce( + loading, + SourceLoaded("smaller", "load-2", SizedSource(100, 80)), + ) + + assert loaded.parameters.exclusions == (CropSpec(60, 70, 40, 10),) + assert cut_exclusions( + loaded.parameters.exclusions, CropSpec(0, 0, 100, 80) + ) == (PixelBounds(60, 70, 100, 80),) diff --git a/tests/jobs/render_support.py b/tests/jobs/render_support.py index 435887c..1aa93c0 100644 --- a/tests/jobs/render_support.py +++ b/tests/jobs/render_support.py @@ -119,9 +119,11 @@ def frozen_segmentation_result(result: np.ndarray) -> np.ndarray: class FakeSegmenter: def __init__(self) -> None: self.calls: list[SegmentRequest] = [] + self.frames: list[np.ndarray] = [] def segment(self, frame: np.ndarray, request: SegmentRequest) -> np.ndarray: self.calls.append(request) + self.frames.append(frame.copy()) result = np.zeros(frame.shape[:2] + (4,), dtype=np.uint8) result[24:104, 32:96, :3] = frame[24:104, 32:96, :3] result[24:104, 32:96, 3] = 255 diff --git a/tests/jobs/test_preview.py b/tests/jobs/test_preview.py index 7dfc7f3..9599221 100644 --- a/tests/jobs/test_preview.py +++ b/tests/jobs/test_preview.py @@ -8,7 +8,8 @@ import pytest from matteloop.core.errors import ErrorCode, ValidationError -from matteloop.core.specs import EdgeMode, FramingSpec, SegmentationSpec +from matteloop.core.geometry import PixelBounds +from matteloop.core.specs import CropSpec, EdgeMode, FramingSpec, SegmentationSpec from matteloop.core.state import JobKind from matteloop.jobs.render import ( FilesystemWorkspacePort, @@ -73,6 +74,55 @@ def test_preview_matches_render_cut_before_global_framing(tmp_path) -> None: assert preview.pre_global_trim_rgba.tobytes() == cut.tobytes() +def test_preview_zeroes_excluded_alpha_in_the_cut_image_and_its_bounds( + tmp_path, +) -> None: + render_request = replace( + request(tmp_path), + framing=FramingSpec( + False, + Decimal("2"), + 0, + Decimal("1"), + exclusions=(CropSpec(32, 24, 32, 80),), + ), + ) + preview = PreviewService( + source=FakeSource(), + segmentation=binding(FakeSegmenter()), + workspace=FilesystemWorkspacePort(), + clock=FakeClock(), + ).preview(render_request, Fraction(0), job(tmp_path, "excluded", JobKind.PREVIEW)) + + with preview.pre_global_trim_rgba.to_image() as cut: + assert cut.getpixel((40, 30)) == (0, 0, 0, 0) + assert cut.getpixel((65, 30)) == (0, 60, 90, 255) + assert preview.local_bounds_estimate == PixelBounds(64, 24, 96, 104) + + +def test_preview_shows_no_filler_pixels_inside_a_model_exclusion( + tmp_path, +) -> None: + render_request = replace( + request(tmp_path), + segmentation=SegmentationSpec( + exclusions=(CropSpec(32, 24, 32, 80),) + ), + ) + + preview = PreviewService( + source=FakeSource(), + segmentation=binding(FakeSegmenter()), + workspace=FilesystemWorkspacePort(), + clock=FakeClock(), + ).preview( + render_request, Fraction(0), job(tmp_path, "model-preview", JobKind.PREVIEW) + ) + + with preview.pre_global_trim_rgba.to_image() as cut: + assert cut.getpixel((40, 30)) == (0, 0, 0, 0) + + def test_preview_fingerprint_tracks_the_prepared_model_weight(tmp_path) -> None: request_with_same_user_settings = request(tmp_path) source = FakeSource() diff --git a/tests/jobs/test_rebuild.py b/tests/jobs/test_rebuild.py index 9ab6224..7f97fe7 100644 --- a/tests/jobs/test_rebuild.py +++ b/tests/jobs/test_rebuild.py @@ -12,6 +12,7 @@ import matteloop.paths as paths_module from matteloop.core.errors import AppError, ErrorCode, ValidationError +from matteloop.core.fingerprints import cut_cache_key from matteloop.core.specs import ( CropSpec, FramingSpec, @@ -19,6 +20,7 @@ OutputSpec, ResizeSpec, SamplingSpec, + SegmentationSpec, TransformSpec, ) from matteloop.core.state import JobKind @@ -34,6 +36,7 @@ RenderService, ) from matteloop.jobs.transform_store import load_transform, transform_sidecar_path +from matteloop.ui.workspace_presentation import request_for_workspace from tests.jobs.render_support import ( ExplodingSource, FakeClock, @@ -414,6 +417,43 @@ def _rebuild_service(workspace: FilesystemWorkspacePort, encoder) -> RenderServi ) +def test_rebuild_of_a_model_exclusion_set_reproduces_its_key_from_the_manifest( + tmp_path, +) -> None: + workspace = FilesystemWorkspacePort() + seed_request = replace( + request(tmp_path), + segmentation=SegmentationSpec( + exclusions=(CropSpec(32, 24, 32, 80),) + ), + ) + original = render_service(workspace=workspace).render( + seed_request, job(tmp_path, "seed-model-rebuild", JobKind.RENDER) + ) + rebuilt_request = request_for_workspace( + original.manifest, + replace(seed_request, segmentation=SegmentationSpec()), + ) + + assert ( + rebuilt_request.segmentation.exclusions + == seed_request.segmentation.exclusions + ) + assert cut_cache_key( + rebuilt_request, + source_sha256=original.manifest.source_sha256, + model_weight_sha256="ab" * 32, + ) == original.manifest.cache_key + + rebuilt = _rebuild_service(workspace, FakeEncoder()).rebuild( + rebuilt_request, + original.cut_workspace, + job(tmp_path, "rebuild-model-set", JobKind.REBUILD), + ) + + assert rebuilt.manifest.cache_key == original.manifest.cache_key + + def test_identity_transform_rebuild_is_byte_identical(tmp_path) -> None: """AC 1: an explicit identity ``TransformSpec`` and the ``RenderRequest`` default must reach the encoder with the same paths and delays and produce @@ -459,6 +499,79 @@ def test_identity_transform_rebuild_is_byte_identical(tmp_path) -> None: assert explicit_delays == default_delays +def test_rebuild_with_a_region_misses_the_cached_union_and_recomputes_it( + tmp_path, +) -> None: + workspace = FilesystemWorkspacePort() + seed_request = replace( + request(tmp_path), + framing=FramingSpec(True, Decimal("2"), 48, Decimal("1")), + ) + original = render_service(workspace=workspace).render( + seed_request, job(tmp_path, "seed-union-region", JobKind.RENDER) + ) + rebuild_request = replace( + seed_request, + rebuild=True, + framing=FramingSpec( + True, + Decimal("2"), + 48, + Decimal("1"), + exclusions=(CropSpec(32, 24, 32, 80),), + ), + output=replace(seed_request.output, filename="region.webp"), + ) + + artifact = _rebuild_service(workspace, FakeEncoder()).rebuild( + rebuild_request, + original.cut_workspace, + job(tmp_path, "rebuild-union-region", JobKind.REBUILD), + ) + + assert (artifact.width, artifact.height) == (128, 176) + metadata = workspace.validate(original.cut_workspace).union_metadata + assert metadata is not None + assert metadata.bounds == (64, 24, 96, 104) + + +def test_rebuild_reuses_union_metadata_when_region_inputs_match(tmp_path) -> None: + workspace = FilesystemWorkspacePort() + seed_request = replace( + request(tmp_path), + framing=FramingSpec( + True, + Decimal("2"), + 48, + Decimal("1"), + exclusions=(CropSpec(32, 24, 32, 80),), + ), + ) + original = render_service(workspace=workspace).render( + seed_request, job(tmp_path, "seed-reusable-region-union", JobKind.RENDER) + ) + + class CountingWorkspace(FilesystemWorkspacePort): + def __init__(self) -> None: + super().__init__() + self.reads = 0 + + def read_cut(self, workspace, index, ownership): + self.reads += 1 + return super().read_cut(workspace, index, ownership) + + counting = CountingWorkspace() + _rebuild_service(counting, FakeEncoder()).rebuild( + replace(seed_request, rebuild=True, output=replace( + seed_request.output, filename="reused-region-union.webp" + )), + original.cut_workspace, + job(tmp_path, "reuse-region-union", JobKind.REBUILD), + ) + + assert counting.reads == original.manifest.frame_count + + def test_trim_keeps_exactly_the_selected_frames_and_their_delays(tmp_path) -> None: """AC 2 / E2 / T7: the kept delays are a *slice* of the full non-uniform grid, not a recomputation over the kept count.""" diff --git a/tests/jobs/test_render.py b/tests/jobs/test_render.py index 7c6ac97..d66c5c6 100644 --- a/tests/jobs/test_render.py +++ b/tests/jobs/test_render.py @@ -48,6 +48,7 @@ RenderService, ValidatedCandidate, ) +from matteloop.jobs.source import DecodedFrame from tests.fixtures.media_factory import make_video from tests.jobs.render_support import ( FakeClock, @@ -55,6 +56,7 @@ FakeEncoder, FakeSegmenter, FakeSource, + FakeSourceInfo, frozen_segmentation_result, job, render_service, @@ -62,6 +64,26 @@ ) +class _OffsetSource(FakeSource): + def probe(self, path, context): + del path, context + self.probe_calls += 1 + return FakeSourceInfo(width=160, height=160) + + def decode(self, path, timestamp, request_id, source_info, context, ownership): + del path, source_info, context + self.decode_calls.append(timestamp) + image = Image.new("RGBA", (160, 160), (0, 60, 90, 255)) + ownership.register(image) + return DecodedFrame( + image, + timestamp, + timestamp, + request_id, + FakeSourceInfo().revision, + ) + + def _validated_candidate( publisher: AtomicOutputPublisher, target: Path, @@ -1519,9 +1541,226 @@ def segment(self, frame, request): assert artifact.manifest.union_metadata is not None -@pytest.mark.parametrize("visible_size", [None, 10]) -def test_invalid_range_framing_fails_after_cut_promotion_and_preserves_output( - tmp_path, visible_size: int | None +def test_render_stores_unexcluded_cuts_but_unions_and_encodes_without_the_region( + tmp_path, +) -> None: + encoder = FakeEncoder() + render_request = replace( + request(tmp_path), + framing=FramingSpec( + False, + Decimal("2"), + 0, + Decimal("1"), + exclusions=(CropSpec(32, 24, 32, 80),), + ), + ) + + artifact = render_service(encoder=encoder).render( + render_request, job(tmp_path, "excluded-render", JobKind.RENDER) + ) + + with artifact.cut_workspace.read_promoted_cut(0) as stored: + assert stored.getpixel((40, 30)) == (0, 60, 90, 255) + with Image.open(artifact.output_path) as output: + output.seek(0) + output.load() + assert output.getpixel((40, 30)) == (0, 0, 0, 0) + assert output.getpixel((65, 30)) == (0, 60, 90, 255) + assert artifact.manifest.union_metadata is not None + assert artifact.manifest.union_metadata.bounds == (64, 24, 96, 104) + + +def test_render_blanks_model_exclusions_and_zeroes_stored_cut( + tmp_path: Path, +) -> None: + segmenter = FakeSegmenter() + render_request = replace( + request(tmp_path), + segmentation=SegmentationSpec( + exclusions=(CropSpec(32, 24, 32, 80),) + ), + ) + + artifact = render_service(segmenter=segmenter).render( + render_request, job(tmp_path, "model-exclusion", JobKind.RENDER) + ) + + assert tuple(segmenter.frames[0][30, 40, :3]) == (124, 116, 104) + with artifact.cut_workspace.read_promoted_cut(0) as stored: + assert stored.getpixel((40, 30)) == (0, 0, 0, 0) + + +def test_render_with_model_exclusions_reads_the_original_pixels_outside_the_box( + tmp_path: Path, +) -> None: + segmenter = FakeSegmenter() + render_request = replace( + request(tmp_path), + segmentation=SegmentationSpec( + exclusions=(CropSpec(32, 24, 32, 80),) + ), + ) + + render_service(segmenter=segmenter).render( + render_request, job(tmp_path, "model-exclusion-edge", JobKind.RENDER) + ) + + frame = segmenter.frames[0] + assert tuple(frame[23, 40, :3]) == (0, 60, 90) + assert tuple(frame[30, 31, :3]) == (0, 60, 90) + assert tuple(frame[104, 40, :3]) == (0, 60, 90) + + +def test_render_with_the_switch_off_sends_the_model_the_unblanked_frame( + tmp_path: Path, +) -> None: + segmenter = FakeSegmenter() + + render_service(segmenter=segmenter).render( + request(tmp_path), job(tmp_path, "model-exclusion-off", JobKind.RENDER) + ) + + assert tuple(segmenter.frames[0][30, 40, :3]) == (0, 60, 90) + + +def test_render_reuses_the_unblanked_cut_set_after_the_switch_is_turned_off( + tmp_path: Path, +) -> None: + segmenter = FakeSegmenter() + service = render_service(segmenter=segmenter) + pre_model = replace( + request(tmp_path), + segmentation=SegmentationSpec( + exclusions=(CropSpec(32, 24, 32, 80),) + ), + ) + post_model = request(tmp_path) + + service.render(pre_model, job(tmp_path, "reuse-pre-first", JobKind.RENDER)) + assert len(segmenter.calls) == 2 + service.render(post_model, job(tmp_path, "reuse-post", JobKind.RENDER)) + assert len(segmenter.calls) == 4 + service.render( + pre_model, job(tmp_path, "reuse-pre-again", JobKind.RENDER) + ) + + assert len(segmenter.calls) == 4 + + +def test_render_with_whole_frame_model_exclusion_notes_untrimmed_canvas( + tmp_path: Path, +) -> None: + render_request = replace( + request(tmp_path), + segmentation=SegmentationSpec( + exclusions=(CropSpec(0, 0, 128, 128),) + ), + framing=FramingSpec(True, Decimal("2"), 0, Decimal("1")), + ) + + artifact = render_service().render( + render_request, job(tmp_path, "whole-model-exclusion", JobKind.RENDER) + ) + + assert (artifact.width, artifact.height) == (128, 128) + assert artifact.manifest.union_metadata is None + assert any("trim skipped" in note for note in artifact.notes) + + +def test_render_maps_offset_source_exclusions_into_cut_coordinates(tmp_path) -> None: + render_request = replace( + request(tmp_path, crop=CropSpec(16, 16, 128, 128)), + framing=FramingSpec( + False, + Decimal("2"), + 0, + Decimal("1"), + exclusions=(CropSpec(48, 40, 16, 16),), + ), + ) + + artifact = render_service(source=_OffsetSource()).render( + render_request, job(tmp_path, "offset-exclusion", JobKind.RENDER) + ) + + with Image.open(artifact.output_path) as output: + output.seek(0) + output.load() + assert output.getpixel((32, 24)) == (0, 0, 0, 0) + assert output.getpixel((48, 40)) == (0, 60, 90, 255) + + +def test_render_peak_rgba_owner_count_is_unchanged_by_exclusions(tmp_path) -> None: + baseline_path = tmp_path / "baseline" + baseline_path.mkdir() + baseline_request = request(baseline_path) + baseline_request.source.write_bytes(b"baseline-video") + baseline = render_service(encoder=FakeEncoder()).render( + baseline_request, + job(baseline_path, "unexcluded-owners", JobKind.RENDER), + ) + encoder = FakeEncoder() + render_request = replace( + request(tmp_path), + framing=FramingSpec( + False, + Decimal("2"), + 0, + Decimal("1"), + exclusions=(CropSpec(32, 24, 32, 80),), + ), + ) + + artifact = render_service(encoder=encoder).render( + render_request, job(tmp_path, "excluded-owners", JobKind.RENDER) + ) + + assert artifact.ownership_peak == baseline.ownership_peak + assert artifact.ownership_current == baseline.ownership_current == 0 + with artifact.cut_workspace.read_promoted_cut(0) as stored: + assert stored.getpixel((40, 30)) == (0, 60, 90, 255) + with Image.open(artifact.output_path) as output: + output.seek(0) + output.load() + assert output.getpixel((40, 30)) == (0, 0, 0, 0) + assert artifact.manifest.union_metadata is not None + assert artifact.manifest.union_metadata.bounds == (64, 24, 96, 104) + + +def test_render_with_trim_and_an_empty_mask_encodes_the_untrimmed_canvas_and_notes_it( + tmp_path, +) -> None: + render_request = replace( + request(tmp_path), + framing=FramingSpec( + True, + Decimal("2"), + 0, + Decimal("1"), + exclusions=(CropSpec(0, 0, 128, 128),), + ), + ) + + artifact = render_service().render( + render_request, job(tmp_path, "empty-exclusion", JobKind.RENDER) + ) + + assert (artifact.width, artifact.height) == (128, 128) + assert artifact.manifest.union_metadata is None + assert ( + "trim skipped: no pixel above the alpha threshold survives the " + "exclusion regions" + ) in artifact.notes + with Image.open(artifact.output_path) as output: + output.seek(0) + output.load() + assert output.getpixel((64, 64)) == (0, 0, 0, 0) + + +@pytest.mark.parametrize("visible_size", [10]) +def test_render_still_refuses_a_trimmed_box_below_the_minimum_dimension( + tmp_path, visible_size: int ) -> None: class SmallOrEmptySegmenter(FakeSegmenter): def segment(self, frame, request): @@ -1544,12 +1783,7 @@ def segment(self, frame, request): render_request, job(tmp_path, "invalid-framing", JobKind.RENDER) ) - expected = ( - ErrorCode.INVALID_FRAMING - if visible_size is None - else ErrorCode.INVALID_FINAL_DIMENSIONS - ) - assert exc.value.code is expected + assert exc.value.code is ErrorCode.INVALID_FINAL_DIMENSIONS assert render_request.output.path.read_bytes() == b"old-output" durable = _only_durable_workspace(render_request.output.directory) assert FilesystemWorkspacePort().validate(durable).frame_count == 2 diff --git a/tests/jobs/test_transform_stage.py b/tests/jobs/test_transform_stage.py index 53925e5..130d0c8 100644 --- a/tests/jobs/test_transform_stage.py +++ b/tests/jobs/test_transform_stage.py @@ -6,10 +6,8 @@ from fractions import Fraction from pathlib import Path -import pytest from PIL import Image -from matteloop.core.errors import ErrorCode, ValidationError from matteloop.core.geometry import FramingPlan, PixelBounds from matteloop.core.rgba import RgbaOwnershipTracker from matteloop.core.specs import CropSpec, FramingSpec, TransformSpec @@ -31,11 +29,11 @@ def test_framing_plan_matches_direct_construction_for_the_identity_framing() -> ) -def test_framing_plan_rejects_trim_without_a_union() -> None: +def test_framing_plan_skips_trim_without_a_union() -> None: framing = FramingSpec(True, Decimal("2"), 0, Decimal("1")) - with pytest.raises(ValidationError) as exc: - framing_plan((200, 180), None, framing) - assert exc.value.code is ErrorCode.INVALID_FRAMING + assert framing_plan((200, 180), None, framing) == FramingPlan( + (200, 180), global_bounds=None, padding=0, stretch_x=Fraction(1) + ) def test_framing_plan_accepts_trim_with_a_union() -> None: @@ -98,6 +96,38 @@ def read_cut(index: int, tracker: RgbaOwnershipTracker) -> Image.Image: assert all(event.overall_total == 6 for event in events) +def test_stage_encoder_frames_zeroes_excluded_boxes_before_framing( + tmp_path: Path, +) -> None: + frame = _colour_frame((255, 0, 0, 255)) + + def read_cut(index: int, tracker: RgbaOwnershipTracker) -> Image.Image: + del index + image = frame.copy() + tracker.register(image) + return image + + tracker = RgbaOwnershipTracker((128, 128)) + paths, _ = stage_encoder_frames( + read_cut, + 1, + FramingPlan((128, 128)), + TransformSpec(), + (100,), + tmp_path / "framed-inputs", + tracker, + job(tmp_path, "excluded", JobKind.RENDER), + (0, 1), + exclusions=(PixelBounds(10, 20, 30, 40),), + ) + + with Image.open(paths[0]) as saved: + saved.load() + assert saved.getpixel((10, 20)) == (0, 0, 0, 0) + assert saved.getpixel((29, 39)) == (0, 0, 0, 0) + assert saved.getpixel((9, 20)) == (255, 0, 0, 255) + + def test_stage_encoder_frames_clamps_a_crop_stored_against_a_larger_frame( tmp_path: Path, ) -> None: diff --git a/tests/jobs/test_workspace.py b/tests/jobs/test_workspace.py index 4c4e27f..a640941 100644 --- a/tests/jobs/test_workspace.py +++ b/tests/jobs/test_workspace.py @@ -2120,6 +2120,64 @@ def test_manifest_rejects_non_authoritative_cache_key_inputs() -> None: assert exc.value.code is ErrorCode.CUT_MANIFEST_INVALID +def test_manifest_accepts_model_exclusions_and_reproduces_the_cut_key( + tmp_path: Path, +) -> None: + inputs = _cache_inputs() + inputs["model_exclusions"] = { + "fill": "imagenet-mean", + "boxes": [[1, 1, 3, 3]], + } + staged, manifest = _completed_staging(tmp_path, inputs=inputs) + decoded = CutManifest.from_json_bytes(manifest.to_json_bytes()) + request = RenderRequest( + source=tmp_path / "source.mp4", + sampling=SamplingSpec(Fraction(0), Fraction(1), 15), + crop=CropSpec(0, 0, 8, 6), + segmentation=SegmentationSpec( + exclusions=(CropSpec(1, 1, 2, 2),) + ), + framing=FramingSpec(), + output=OutputSpec(tmp_path, "output.webp"), + ) + + assert decoded == manifest + assert decoded.cache_key == cut_cache_key( + request, + source_sha256="a" * 64, + model_weight_sha256="b" * 64, + ) + assert staged.cache_key == decoded.cache_key + + +@pytest.mark.parametrize( + "box", + [ + [2, 1, 1, 2], + [-1, 0, 1, 1], + ["1", 0, 1, 1], + ], + ids=["inverted", "negative", "non-integer"], +) +def test_manifest_rejects_malformed_model_exclusion_boxes( + box: list[object], +) -> None: + inputs = _cache_inputs() + inputs["model_exclusions"] = { + "fill": "imagenet-mean", + "boxes": [box], + } + + with pytest.raises(AppError) as exc: + CutManifest.cache_key_for(inputs) + + assert exc.value.code is ErrorCode.CUT_MANIFEST_INVALID + + +def test_manifest_without_model_exclusions_still_validates() -> None: + assert CutManifest.cache_key_for(_cache_inputs()) + + def test_manifest_cache_key_matches_the_task_4_authoritative_fingerprint( tmp_path: Path, ) -> None: diff --git a/tests/ui/test_crop_canvas.py b/tests/ui/test_crop_canvas.py index d7af6d4..88a8637 100644 --- a/tests/ui/test_crop_canvas.py +++ b/tests/ui/test_crop_canvas.py @@ -2,17 +2,25 @@ import pytest from PySide6.QtCore import QPoint, Qt -from PySide6.QtGui import QImage +from PySide6.QtGui import QColor, QImage +from PySide6.QtWidgets import QPushButton, QVBoxLayout, QWidget from matteloop.core.crop_state import CropChanged +from matteloop.core.exclusion import cut_exclusions from matteloop.core.geometry import MediaTransform, PointF +from matteloop.core.parameters import ExclusionsChanged from matteloop.core.specs import CropSpec from matteloop.ui.crop_canvas import CropCanvas from matteloop.ui.crop_presentation import CropPresentation +from matteloop.ui.exclusion_canvas import ExclusionCanvas from matteloop.ui.preview_canvas import PreviewStage -def _presentation(crop: CropSpec = CropSpec(10, 10, 40, 20)) -> CropPresentation: +def _presentation( + crop: CropSpec = CropSpec(10, 10, 40, 20), + exclusions: tuple[CropSpec, ...] = (), + exclusions_before_model: bool = False, +) -> CropPresentation: return CropPresentation( source_id="source", width=100, @@ -22,19 +30,31 @@ def _presentation(crop: CropSpec = CropSpec(10, 10, 40, 20)) -> CropPresentation rotation=0, pixel_aspect=1, crop=crop, + exclusions=exclusions, + exclusions_before_model=exclusions_before_model, ) -def _canvas(qtbot) -> CropCanvas: - canvas = CropCanvas() +def _canvas( + qtbot, crop: CropSpec = CropSpec(10, 10, 40, 20) +) -> CropCanvas: + canvas = ExclusionCanvas() qtbot.addWidget(canvas) canvas.resize(200, 100) canvas.set_frame(QImage(100, 50, QImage.Format.Format_RGBA8888)) - canvas.apply_presentation(_presentation(), active=True, editable=True) + canvas.apply_presentation( + _presentation(crop=crop), active=True, editable=True + ) canvas.show() return canvas +def _source_pos(canvas: CropCanvas, x: int, y: int) -> QPoint: + assert canvas._geometry is not None + point = canvas._geometry.transform.source_to_widget(PointF(x, y)) + return QPoint(round(point.x), round(point.y)) + + def test_crop_canvas_drag_emits_oriented_crop_changes_from_shared_geometry( qtbot, ) -> None: @@ -50,6 +70,710 @@ def test_crop_canvas_drag_emits_oriented_crop_changes_from_shared_geometry( assert events[-1].crop == CropSpec(22, 16, 40, 20) +def test_a_region_drag_dispatches_exactly_once_on_release(qtbot) -> None: + canvas = _canvas(qtbot) + canvas.apply_presentation( + _presentation(exclusions=(CropSpec(20, 10, 30, 20),)), + active=True, + editable=True, + ) + canvas.set_exclusion_edit(True) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.mousePress( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 25, 15), + ) + for x in range(26, 46): + qtbot.mouseMove(canvas, _source_pos(canvas, x, 15)) + + assert events == [] + qtbot.mouseRelease( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 45, 15), + ) + + assert len(events) == 1 + assert events[0] == ExclusionsChanged((CropSpec(40, 10, 30, 20),)) + + +@pytest.mark.parametrize( + ("index", "updated"), + [ + (0, (CropSpec(60, 10, 20, 20),)), + (1, (CropSpec(20, 10, 20, 20),)), + (0, (CropSpec(30, 10, 20, 20), CropSpec(60, 10, 20, 20))), + ], +) +def test_region_state_change_cancels_an_in_flight_drag( + qtbot, index: int, updated: tuple[CropSpec, ...] +) -> None: + regions = (CropSpec(20, 10, 20, 20), CropSpec(60, 10, 20, 20)) + canvas = _canvas(qtbot) + canvas.apply_presentation( + _presentation(exclusions=regions), active=True, editable=True + ) + canvas.set_exclusion_edit(True) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + x = 25 if index == 0 else 65 + qtbot.mousePress(canvas, Qt.MouseButton.LeftButton, pos=_source_pos(canvas, x, 15)) + qtbot.mouseMove(canvas, _source_pos(canvas, x + 5, 15)) + canvas.apply_presentation( + _presentation(exclusions=updated), active=True, editable=True + ) + + qtbot.mouseRelease( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, x + 5, 15), + ) + + assert events == [] + assert canvas._drag_exclusion is None # noqa: SLF001 + + +def test_creating_a_region_cancels_the_previous_drag_before_nudging_it(qtbot) -> None: + original = CropSpec(20, 10, 20, 20) + created = CropSpec(40, 20, 20, 10) + canvas = _canvas(qtbot) + canvas.apply_presentation( + _presentation(exclusions=(original,)), active=True, editable=True + ) + canvas.set_exclusion_edit(True) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.mousePress( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 25, 15), + ) + qtbot.mouseMove(canvas, _source_pos(canvas, 30, 15)) + canvas.setFocus() + qtbot.waitUntil(canvas.hasFocus, timeout=1000) + qtbot.keyClick(canvas, Qt.Key.Key_N) + + assert events == [ExclusionsChanged((original, created))] + canvas.apply_presentation( + _presentation(exclusions=(original, created)), + active=True, + editable=True, + ) + assert canvas._selected_exclusion == 1 # noqa: SLF001 + assert canvas._drag_exclusion is None # noqa: SLF001 + + qtbot.keyClick(canvas, Qt.Key.Key_Right) + qtbot.mouseRelease( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 30, 15), + ) + + assert events == [ + ExclusionsChanged((original, created)), + ExclusionsChanged((original, CropSpec(41, 20, 20, 10))), + ] + + +def test_exclusion_editing_stays_active_when_crop_is_disabled(qtbot) -> None: + canvas = _canvas(qtbot) + canvas.apply_presentation( + _presentation(), active=False, editable=True + ) + canvas.set_exclusion_edit(True) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.mousePress( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 10, 10), + ) + qtbot.mouseMove(canvas, _source_pos(canvas, 30, 25)) + qtbot.mouseRelease( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 30, 25), + ) + + assert events == [ExclusionsChanged((CropSpec(10, 10, 20, 15),))] + + +def test_persisted_exclusions_are_painted_when_crop_is_disabled(qtbot) -> None: + canvas = _canvas(qtbot) + frame = QImage(100, 50, QImage.Format.Format_RGBA8888) + frame.fill(QColor("#ffffff")) + canvas.set_frame(frame) + canvas.apply_presentation( + _presentation(exclusions=(CropSpec(20, 10, 30, 20),)), + active=False, + editable=True, + ) + canvas.show() + qtbot.wait(10) + + color = canvas.grab().toImage().pixelColor(_source_pos(canvas, 30, 20)) + + assert color.red() > color.green() + + +def test_partly_outside_exclusion_paints_only_the_cut_intersection(qtbot) -> None: + crop = CropSpec(10, 10, 60, 30) + region = CropSpec(60, 20, 20, 10) + canvas = _canvas(qtbot, crop) + frame = QImage(100, 50, QImage.Format.Format_RGBA8888) + frame.fill(QColor("#ffffff")) + canvas.set_frame(frame) + canvas.apply_presentation( + _presentation(crop=crop, exclusions=(region,)), + active=False, + editable=True, + ) + canvas.show() + qtbot.wait(10) + + boxes = cut_exclusions((region,), crop) + assert len(boxes) == 1 + box = boxes[0] + assert canvas._effective_exclusion(region) == CropSpec( # noqa: SLF001 + crop.x + box.left, + crop.y + box.top, + box.width, + box.height, + ) + + image = canvas.grab().toImage() + effective = image.pixelColor(_source_pos(canvas, 65, 25)) + ineffective = image.pixelColor(_source_pos(canvas, 75, 25)) + assert effective.red() > effective.green() + assert ineffective.red() > ineffective.green() + assert effective != ineffective + + +def test_wholly_outside_exclusion_is_faded_and_has_no_effective_part(qtbot) -> None: + crop = CropSpec(10, 10, 60, 30) + inside = CropSpec(20, 15, 10, 10) + outside = CropSpec(75, 20, 10, 10) + canvas = _canvas(qtbot, crop) + frame = QImage(100, 50, QImage.Format.Format_RGBA8888) + frame.fill(QColor("#ffffff")) + canvas.set_frame(frame) + canvas.apply_presentation( + _presentation(crop=crop, exclusions=(inside, outside)), + active=False, + editable=True, + ) + canvas.show() + qtbot.wait(10) + + assert cut_exclusions((outside,), crop) == () + assert canvas._effective_exclusion(outside) is None # noqa: SLF001 + image = canvas.grab().toImage() + assert image.pixelColor(_source_pos(canvas, 20, 20)) != image.pixelColor( + _source_pos(canvas, 80, 25) + ) + + +def test_wholly_inside_exclusion_keeps_the_solid_treatment(qtbot) -> None: + crop = CropSpec(10, 10, 60, 30) + region = CropSpec(20, 15, 10, 10) + canvas = _canvas(qtbot, crop) + frame = QImage(100, 50, QImage.Format.Format_RGBA8888) + frame.fill(QColor("#ffffff")) + canvas.set_frame(frame) + canvas.apply_presentation( + _presentation(crop=crop, exclusions=(region,)), + active=False, + editable=True, + ) + canvas.show() + qtbot.wait(10) + + box = cut_exclusions((region,), crop)[0] + assert canvas._effective_exclusion(region) == CropSpec( # noqa: SLF001 + crop.x + box.left, + crop.y + box.top, + box.width, + box.height, + ) + color = canvas.grab().toImage().pixelColor(_source_pos(canvas, 25, 20)) + assert color.red() > color.green() + + +def test_an_abandoned_rubber_band_dispatches_nothing(qtbot) -> None: + canvas = _canvas(qtbot) + canvas.set_exclusion_edit(True) + canvas.activateWindow() + canvas.setFocus() + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.mousePress( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 10, 10), + ) + qtbot.mouseMove(canvas, _source_pos(canvas, 30, 25)) + canvas.setFocus() + qtbot.waitUntil(canvas.hasFocus, timeout=1000) + assert canvas._rubber_start is not None + qtbot.keyClick(canvas, Qt.Key.Key_Escape) + qtbot.mouseRelease( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 30, 25), + ) + + assert events == [] + + +def test_region_mode_rubber_band_adds_a_region_on_release(qtbot) -> None: + canvas = _canvas(qtbot) + canvas.set_exclusion_edit(True) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.mousePress( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 10, 10), + ) + for x in range(11, 31): + y = round(10 + (x - 10) * 15 / 20) + qtbot.mouseMove(canvas, _source_pos(canvas, x, y)) + + assert events == [] + qtbot.mouseRelease( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 30, 25), + ) + + assert events == [ExclusionsChanged((CropSpec(10, 10, 20, 15),))] + + +def test_region_mode_discards_a_rubber_band_below_eight_pixels(qtbot) -> None: + canvas = _canvas(qtbot) + canvas.set_exclusion_edit(True) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.mousePress( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 10, 10), + ) + qtbot.mouseMove(canvas, _source_pos(canvas, 17, 18)) + qtbot.mouseRelease( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 17, 18), + ) + + assert events == [] + + +def test_region_mode_gives_the_crop_no_hit_targets(qtbot) -> None: + canvas = _canvas(qtbot) + canvas.apply_presentation( + _presentation(exclusions=(CropSpec(70, 30, 20, 10),)), + active=True, + editable=True, + ) + canvas.set_exclusion_edit(True) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.mousePress( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 10, 10), + ) + for x in range(11, 31): + y = round(10 + (x - 10) * 15 / 20) + qtbot.mouseMove(canvas, _source_pos(canvas, x, y)) + + assert events == [] + assert canvas._dragged is None + assert canvas._rubber_start is not None + qtbot.mouseRelease( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 30, 25), + ) + + assert events == [ + ExclusionsChanged( + ( + CropSpec(70, 30, 20, 10), + CropSpec(10, 10, 20, 15), + ) + ) + ] + assert canvas._dragged is None + + +def test_click_selects_a_region_and_the_same_drag_moves_it(qtbot) -> None: + canvas = _canvas(qtbot) + canvas.apply_presentation( + _presentation(exclusions=(CropSpec(20, 10, 30, 20),)), + active=True, + editable=True, + ) + canvas.set_exclusion_edit(True) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.mousePress( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 25, 15), + ) + qtbot.mouseMove(canvas, _source_pos(canvas, 35, 15)) + qtbot.mouseRelease( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 35, 15), + ) + + assert events == [ExclusionsChanged((CropSpec(30, 10, 30, 20),))] + + +def test_resizing_a_region_dispatches_exactly_once_on_release(qtbot) -> None: + canvas = _canvas(qtbot) + region = CropSpec(20, 10, 30, 20) + canvas.apply_presentation( + _presentation(exclusions=(region,)), active=True, editable=True + ) + canvas.set_exclusion_edit(True) + qtbot.mouseClick( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 25, 15), + ) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.mousePress( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 50, 20), + ) + for x in range(51, 61): + qtbot.mouseMove(canvas, _source_pos(canvas, x, 20)) + + assert events == [] + qtbot.mouseRelease( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 60, 20), + ) + + assert events == [ExclusionsChanged((CropSpec(20, 10, 40, 20),))] + + +def test_delete_removes_the_selected_region(qtbot) -> None: + canvas = _canvas(qtbot) + canvas.apply_presentation( + _presentation(exclusions=(CropSpec(20, 10, 30, 20),)), + active=True, + editable=True, + ) + canvas.set_exclusion_edit(True) + canvas.activateWindow() + canvas.setFocus() + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.mouseClick( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 25, 15), + ) + canvas.setFocus() + qtbot.waitUntil(canvas.hasFocus, timeout=1000) + assert canvas._selected_exclusion == 0 + qtbot.keyClick(canvas, Qt.Key.Key_Delete) + + assert events == [ExclusionsChanged(())] + + +def test_arrow_key_nudges_the_selected_region_once(qtbot) -> None: + canvas = _canvas(qtbot) + canvas.apply_presentation( + _presentation(exclusions=(CropSpec(20, 10, 30, 20),)), + active=True, + editable=True, + ) + canvas.set_exclusion_edit(True) + qtbot.mouseClick( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 25, 15), + ) + canvas.setFocus() + qtbot.waitUntil(canvas.hasFocus, timeout=1000) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.keyClick(canvas, Qt.Key.Key_Right) + + assert events == [ExclusionsChanged((CropSpec(21, 10, 30, 20),))] + + +def test_arrow_key_can_move_a_region_onto_the_crop_bounds(qtbot) -> None: + canvas = _canvas(qtbot) + region = CropSpec(19, 10, 30, 20) + canvas.apply_presentation( + _presentation(crop=CropSpec(20, 10, 30, 20), exclusions=(region,)), + active=True, + editable=True, + ) + canvas.set_exclusion_edit(True) + qtbot.mouseClick( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 25, 15), + ) + canvas.setFocus() + qtbot.waitUntil(canvas.hasFocus, timeout=1000) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.keyClick(canvas, Qt.Key.Key_Right) + + assert events == [ExclusionsChanged((CropSpec(20, 10, 30, 20),))] + + +def test_region_mode_keyboard_creates_a_region(qtbot) -> None: + canvas = _canvas(qtbot) + canvas.set_exclusion_edit(True) + canvas.setFocus() + qtbot.waitUntil(canvas.hasFocus, timeout=1000) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.keyClick(canvas, Qt.Key.Key_Insert) + + assert len(events) == 1 + assert isinstance(events[0], ExclusionsChanged) + assert len(events[0].exclusions) == 1 + + +def test_region_mode_bracket_keys_cycle_selection_in_focus_chain(qtbot) -> None: + regions = (CropSpec(20, 10, 20, 20), CropSpec(60, 10, 20, 20)) + window = QWidget() + qtbot.addWidget(window) + layout = QVBoxLayout(window) + canvas = ExclusionCanvas(window) + button = QPushButton("Other focusable control", window) + layout.addWidget(canvas) + layout.addWidget(button) + canvas.resize(200, 100) + canvas.set_frame(QImage(100, 50, QImage.Format.Format_RGBA8888)) + canvas.apply_presentation( + _presentation(exclusions=regions), active=True, editable=True + ) + canvas.set_exclusion_edit(True) + window.setTabOrder(canvas, button) + window.resize(240, 240) + window.show() + canvas.setFocus() + qtbot.waitUntil(canvas.hasFocus, timeout=1000) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.keyClick(canvas, Qt.Key.Key_BracketRight) + assert canvas._selected_exclusion == 0 # noqa: SLF001 + assert canvas.hasFocus() + qtbot.keyClick(canvas, Qt.Key.Key_BracketLeft) + assert canvas._selected_exclusion == 1 # noqa: SLF001 + qtbot.keyClick(canvas, Qt.Key.Key_Delete) + + assert events == [ExclusionsChanged((regions[0],))] + + +def test_crop_controls_are_hidden_when_crop_is_disabled_but_regions_are_painted( + qtbot, +) -> None: + canvas = _canvas(qtbot) + frame = QImage(100, 50, QImage.Format.Format_RGBA8888) + frame.fill(QColor("#ffffff")) + canvas.set_frame(frame) + canvas.apply_presentation( + _presentation(exclusions=(CropSpec(20, 10, 30, 20),)), + active=False, + editable=True, + ) + qtbot.wait(10) + + image = canvas.grab().toImage() + accent = QColor("#B7F34A").rgba() + + assert any( + image.pixelColor(x, y).red() > image.pixelColor(x, y).green() + for x in range(image.width()) + for y in range(image.height()) + ) + assert not any( + image.pixelColor(x, y).rgba() == accent + for x in range(image.width()) + for y in range(image.height()) + ) + + +def test_region_mode_announces_mode_without_a_selection(qtbot) -> None: + canvas = _canvas(qtbot) + canvas.set_exclusion_edit(True) + + assert canvas.accessibleDescription() == ( + "Exclusion region mode: 0 region(s); no region selected" + ) + + +def test_leaving_region_mode_restores_crop_handles_and_keeps_painting_regions( + qtbot, +) -> None: + region = CropSpec(20, 10, 30, 20) + canvas = _canvas(qtbot) + canvas.apply_presentation( + _presentation(exclusions=(region,)), active=True, editable=True + ) + frame = QImage(100, 50, QImage.Format.Format_RGBA8888) + frame.fill(QColor("#123456")) + canvas.set_frame(frame) + canvas.set_exclusion_edit(True) + qtbot.wait(10) + region_mode_image = canvas.grab().toImage() + assert canvas._geometry is not None + handle = canvas._geometry.visual["north_west"] + handle_box = ( + max(0, round(handle.left)), + max(0, round(handle.top)), + min(region_mode_image.width(), round(handle.right)), + min(region_mode_image.height(), round(handle.bottom)), + ) + + def accent_pixels(image: QImage) -> int: + accent = QColor("#B7F34A").rgba() + return sum( + image.pixelColor(x, y).rgba() == accent + for x in range(handle_box[0], handle_box[2] + 1) + for y in range(handle_box[1], handle_box[3] + 1) + ) + + region_point = _source_pos(canvas, 35, 20) + region_pixel = region_mode_image.pixelColor(region_point) + assert accent_pixels(region_mode_image) == 0 + + qtbot.mouseClick( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 25, 15), + ) + canvas.set_exclusion_edit(False) + qtbot.wait(10) + crop_mode_image = canvas.grab().toImage() + + assert canvas._selected_exclusion is None + assert canvas._painted_exclusions() == (region,) + assert accent_pixels(crop_mode_image) > 0 + assert crop_mode_image.pixelColor(region_point) == region_pixel + + +def test_region_mode_announces_the_selected_region_bounds(qtbot) -> None: + canvas = _canvas(qtbot) + canvas.apply_presentation( + _presentation(exclusions=(CropSpec(20, 10, 30, 20),)), + active=True, + editable=True, + ) + canvas.set_exclusion_edit(True) + + qtbot.mouseClick( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, 25, 15), + ) + + assert canvas.accessibleDescription() == ( + "Exclusion region 1 of 1: x 20, y 10, width 30, height 20 source pixels" + ) + + +@pytest.mark.parametrize( + ("region", "status"), + [ + (CropSpec(60, 20, 20, 10), "partly outside crop"), + (CropSpec(75, 20, 10, 10), "entirely outside crop"), + ], + ids=["partly-outside", "wholly-outside"], +) +def test_selected_exclusion_announces_when_crop_does_not_apply( + qtbot, region: CropSpec, status: str +) -> None: + crop = CropSpec(10, 10, 60, 30) + canvas = _canvas(qtbot, crop) + canvas.apply_presentation( + _presentation(crop=crop, exclusions=(region,)), + active=True, + editable=True, + ) + canvas.set_exclusion_edit(True) + + qtbot.mouseClick( + canvas, + Qt.MouseButton.LeftButton, + pos=_source_pos(canvas, region.x + 1, region.y + 1), + ) + + assert canvas.accessibleDescription().endswith(f"; {status}") + + +def test_model_exclusions_are_painted_hatched_and_announced(qtbot) -> None: + region = CropSpec(20, 10, 30, 20) + canvas = _canvas(qtbot) + frame = QImage(100, 50, QImage.Format.Format_RGBA8888) + frame.fill(QColor("#ffffff")) + canvas.set_frame(frame) + canvas.apply_presentation( + _presentation( + exclusions=(region,), exclusions_before_model=True + ), + active=True, + editable=True, + ) + canvas.show() + canvas.set_exclusion_edit(True) + qtbot.wait(10) + + image = canvas.grab().toImage() + assert canvas._geometry is not None # noqa: SLF001 + rect = canvas._widget_rect(canvas._geometry, region) # noqa: SLF001 + left = round(rect.left()) + right = round(rect.right()) + top = round(rect.top()) + bottom = round(rect.bottom()) + colors = { + image.pixelColor(x, y).rgba() + for x in range(left + 2, right - 2) + for y in range(top + 2, bottom - 2) + } + + assert len(colors) > 1 + assert any( + (color := image.pixelColor(x, y)).red() > color.green() + for x in range(left + 2, right - 2) + for y in range(top + 2, bottom - 2) + ) + assert canvas.accessibleDescription().endswith("; blanked before the model") + + def test_crop_canvas_keyboard_moves_overlay_by_one_source_pixel(qtbot) -> None: canvas = _canvas(qtbot) events: list[object] = [] diff --git a/tests/ui/test_exclusion_context_menu.py b/tests/ui/test_exclusion_context_menu.py new file mode 100644 index 0000000..d4e44ed --- /dev/null +++ b/tests/ui/test_exclusion_context_menu.py @@ -0,0 +1,312 @@ +from __future__ import annotations + +from PySide6.QtCore import QPoint, Qt, QTimer +from PySide6.QtGui import QContextMenuEvent, QImage +from PySide6.QtWidgets import QMenu + +from matteloop.core.geometry import PointF +from matteloop.core.parameters import ExclusionsChanged +from matteloop.core.specs import CropSpec +from matteloop.ui.crop_presentation import CropPresentation +from matteloop.ui.exclusion_canvas import ExclusionCanvas + + +def _presentation( + exclusions: tuple[CropSpec, ...] = (), +) -> CropPresentation: + return CropPresentation( + source_id="source", + width=100, + height=50, + coded_width=100, + coded_height=50, + rotation=0, + pixel_aspect=1, + crop=CropSpec(10, 10, 80, 30), + exclusions=exclusions, + ) + + +def _canvas( + qtbot, + exclusions: tuple[CropSpec, ...] = (), + *, + editable: bool = True, +) -> ExclusionCanvas: + canvas = ExclusionCanvas() + qtbot.addWidget(canvas) + canvas.resize(200, 100) + canvas.set_frame(QImage(100, 50, QImage.Format.Format_RGBA8888)) + canvas.apply_presentation( + _presentation(exclusions), active=True, editable=editable + ) + canvas.show() + return canvas + + +def _source_pos(canvas: ExclusionCanvas, x: int, y: int) -> QPoint: + assert canvas._geometry is not None # noqa: SLF001 + point = canvas._geometry.transform.source_to_widget(PointF(x, y)) + return QPoint(round(point.x), round(point.y)) + + +def _action(menu: QMenu, text: str): + return next(action for action in menu.actions() if action.text() == text) + + +def test_context_menu_lists_region_actions_for_empty_and_occupied_canvas_areas( + qtbot, +) -> None: + empty_canvas = _canvas(qtbot) + empty_menu = empty_canvas._build_context_menu( # noqa: SLF001 + _source_pos(empty_canvas, 5, 5) + ) + + occupied_canvas = _canvas( + qtbot, exclusions=(CropSpec(20, 10, 30, 20),) + ) + occupied_menu = occupied_canvas._build_context_menu( # noqa: SLF001 + _source_pos(occupied_canvas, 25, 15) + ) + + assert [action.text() for action in empty_menu.actions()] == [ + "Exclude this area", + "Edit exclusion regions", + ] + assert [action.text() for action in occupied_menu.actions()] == [ + "Remove this region", + "Remove all regions", + "Edit exclusion regions", + ] + # A QAction carries no accessible-name API; assistive technology reads the + # action's text, so that is what has to be present and meaningful. + assert all(action.text().strip() for action in occupied_menu.actions()) + assert all(not action.isSeparator() for action in occupied_menu.actions()) + assert _action(occupied_menu, "Edit exclusion regions").isCheckable() + assert not _action(occupied_menu, "Edit exclusion regions").isChecked() + + occupied_canvas.set_exclusion_edit(True) + checked_menu = occupied_canvas._build_context_menu( # noqa: SLF001 + _source_pos(occupied_canvas, 25, 15) + ) + assert _action(checked_menu, "Edit exclusion regions").isChecked() + + +def test_context_menu_excludes_a_clamped_default_region_with_one_dispatch( + qtbot, +) -> None: + canvas = _canvas(qtbot) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + menu = canvas._build_context_menu( # noqa: SLF001 + _source_pos(canvas, 95, 45) + ) + _action(menu, "Exclude this area").trigger() + + assert events == [ExclusionsChanged((CropSpec(80, 40, 20, 10),))] + assert canvas._exclusion_edit is True # noqa: SLF001 + assert canvas._selected_exclusion == 0 # noqa: SLF001 + + +def test_context_menu_removes_the_region_under_the_pointer(qtbot) -> None: + regions = (CropSpec(10, 5, 20, 15), CropSpec(70, 20, 20, 15)) + canvas = _canvas(qtbot, exclusions=regions) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + menu = canvas._build_context_menu( # noqa: SLF001 + _source_pos(canvas, 75, 25) + ) + _action(menu, "Remove this region").trigger() + + assert events == [ExclusionsChanged((regions[0],))] + + +def test_context_menu_removes_all_regions_with_one_dispatch(qtbot) -> None: + canvas = _canvas( + qtbot, exclusions=(CropSpec(10, 5, 20, 15), CropSpec(70, 20, 20, 15)) + ) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + menu = canvas._build_context_menu( # noqa: SLF001 + _source_pos(canvas, 5, 5) + ) + _action(menu, "Remove all regions").trigger() + + assert events == [ExclusionsChanged(())] + + +class _ContextMenuProbe(ExclusionCanvas): + def __init__(self, *, trigger_text: str | None = None) -> None: + super().__init__() + self.shown_position: QPoint | None = None + self.shown_actions: list[str] = [] + self.trigger_text = trigger_text + + def _show_context_menu(self, menu: QMenu, position: QPoint) -> None: + self.shown_position = position + self.shown_actions = [action.text() for action in menu.actions()] + if self.trigger_text is not None: + _action(menu, self.trigger_text).trigger() + + +def test_keyboard_context_menu_uses_the_focused_region_position(qtbot) -> None: + canvas = _ContextMenuProbe() + qtbot.addWidget(canvas) + canvas.resize(200, 100) + canvas.set_frame(QImage(100, 50, QImage.Format.Format_RGBA8888)) + canvas.apply_presentation( + _presentation(exclusions=(CropSpec(20, 10, 30, 20),)), + active=True, + editable=True, + ) + canvas.show() + canvas.set_exclusion_edit(True) + canvas._set_exclusion_selection(0) # noqa: SLF001 + canvas.activateWindow() + canvas.setFocus(Qt.FocusReason.OtherFocusReason) + qtbot.waitUntil(canvas.hasFocus, timeout=1000) + + assert canvas._geometry is not None # noqa: SLF001 + expected = canvas._widget_rect( # noqa: SLF001 + canvas._geometry, canvas._presentation.exclusions[0] # noqa: SLF001 + ) + qtbot.keyClick(canvas, Qt.Key.Key_Menu) + + assert canvas.shown_position is not None + assert canvas.shown_position == QPoint( + round(expected.center().x()), round(expected.center().y()) + ) + assert canvas.shown_actions[0] == "Remove this region" + + +def test_keyboard_context_menu_removes_the_selected_region_when_regions_overlap( + qtbot, +) -> None: + regions = (CropSpec(10, 5, 40, 30), CropSpec(20, 10, 30, 20)) + canvas = _ContextMenuProbe(trigger_text="Remove this region") + qtbot.addWidget(canvas) + canvas.resize(200, 100) + canvas.set_frame(QImage(100, 50, QImage.Format.Format_RGBA8888)) + canvas.apply_presentation( + _presentation(exclusions=regions), active=True, editable=True + ) + canvas.show() + canvas.set_exclusion_edit(True) + canvas._set_exclusion_selection(0) # noqa: SLF001 + canvas.activateWindow() + canvas.setFocus(Qt.FocusReason.OtherFocusReason) + qtbot.waitUntil(canvas.hasFocus, timeout=1000) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + qtbot.keyClick(canvas, Qt.Key.Key_Menu) + + assert events == [ExclusionsChanged((regions[1],))] + + +def test_context_menu_actions_revalidate_editability_at_activation(qtbot) -> None: + canvas = _canvas(qtbot) + empty_menu = canvas._build_context_menu( # noqa: SLF001 + _source_pos(canvas, 5, 5) + ) + events: list[object] = [] + canvas.command_requested.connect(events.append) + canvas.apply_presentation(_presentation(), active=True, editable=False) + + _action(empty_menu, "Exclude this area").trigger() + _action(empty_menu, "Edit exclusion regions").trigger() + + assert events == [] + assert canvas._exclusion_edit is False # noqa: SLF001 + + regions = (CropSpec(20, 10, 30, 20),) + canvas.apply_presentation( + _presentation(exclusions=regions), active=True, editable=True + ) + region_menu = canvas._build_context_menu( # noqa: SLF001 + _source_pos(canvas, 25, 15) + ) + canvas.apply_presentation( + _presentation(exclusions=regions), active=True, editable=False + ) + _action(region_menu, "Remove this region").trigger() + _action(region_menu, "Remove all regions").trigger() + + assert events == [] + + +def test_context_menu_removal_revalidates_the_region_identity(qtbot) -> None: + regions = (CropSpec(20, 10, 30, 20), CropSpec(70, 20, 20, 15)) + canvas = _canvas(qtbot, exclusions=regions) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + menu = canvas._build_context_menu( # noqa: SLF001 + _source_pos(canvas, 25, 15) + ) + canvas.apply_presentation( + _presentation(exclusions=(regions[1],)), active=True, editable=True + ) + _action(menu, "Remove this region").trigger() + + assert events == [] + + +def test_context_menu_removal_rejects_a_gone_duplicate_region(qtbot) -> None: + first = CropSpec(20, 10, 30, 20) + second = CropSpec(20, 10, 30, 20) + canvas = _canvas(qtbot, exclusions=(first, second)) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + menu = canvas._build_context_menu( # noqa: SLF001 + _source_pos(canvas, 25, 15) + ) + canvas.apply_presentation( + _presentation(exclusions=(first,)), active=True, editable=True + ) + _action(menu, "Remove this region").trigger() + + assert events == [] + + +def test_context_menu_is_deleted_after_each_opening(qtbot) -> None: + canvas = _canvas(qtbot) + opened_counts: list[int] = [] + + def dismiss_open_menu() -> None: + menus = canvas.findChildren(QMenu) + visible_menus = [menu for menu in menus if menu.isVisible()] + opened_counts.append(len(visible_menus)) + for menu in visible_menus: + menu.close() + + for index in range(4): + event = QContextMenuEvent( + QContextMenuEvent.Reason.Mouse, + _source_pos(canvas, 5, 5), + _source_pos(canvas, 5, 5), + ) + QTimer.singleShot(0, dismiss_open_menu) + canvas.contextMenuEvent(event) + qtbot.waitUntil(lambda: len(opened_counts) == index + 1, timeout=1000) + qtbot.wait(1) + assert canvas.findChildren(QMenu) == [] + + assert opened_counts == [1, 1, 1, 1] + + +def test_context_menu_offers_no_actions_when_editing_is_unavailable(qtbot) -> None: + canvas = _canvas(qtbot, exclusions=(CropSpec(20, 10, 30, 20),), editable=False) + events: list[object] = [] + canvas.command_requested.connect(events.append) + + menu = canvas._build_context_menu( # noqa: SLF001 + _source_pos(canvas, 25, 15) + ) + + assert menu.actions() == [] + assert events == [] diff --git a/tests/ui/test_main_window_state_and_source_drop.py b/tests/ui/test_main_window_state_and_source_drop.py index f2f96c6..47bfdcb 100644 --- a/tests/ui/test_main_window_state_and_source_drop.py +++ b/tests/ui/test_main_window_state_and_source_drop.py @@ -2,15 +2,20 @@ from collections.abc import Callable from dataclasses import dataclass, replace +from fractions import Fraction from pathlib import Path import pytest -from PySide6.QtCore import QByteArray, QMimeData, QSettings, Qt, QUrl +from PySide6.QtCore import QByteArray, QMimeData, QPoint, QSettings, Qt, QUrl from PySide6.QtGui import QDragEnterEvent, QDragLeaveEvent, QDropEvent, QFontMetrics from PySide6.QtTest import QTest from PySide6.QtWidgets import QApplication, QLabel, QPushButton from matteloop.core.errors import AppError, ErrorCode +from matteloop.core.exclusion import cut_exclusions +from matteloop.core.geometry import PointF +from matteloop.core.parameters import ExclusionsChanged +from matteloop.core.specs import CropSpec from matteloop.core.state import ( AppState, ArtifactResult, @@ -60,6 +65,14 @@ def dispatch(self, command: object) -> None: self.commands.append(command) +@dataclass(frozen=True) +class _SourceMetadata: + width: int = 100 + height: int = 50 + duration: Fraction = Fraction(2) + average_rate: Fraction = Fraction(30) + + def _ready() -> AppState: return reduce( reduce(AppState(), SourceLoadRequested("source", "load")), @@ -67,6 +80,14 @@ def _ready() -> AppState: ) +def _ready_with_crop() -> AppState: + state = reduce( + reduce(AppState(), SourceLoadRequested("source", "load")), + SourceLoaded("source", "load", _SourceMetadata()), + ) + return replace(state, crop=CropSpec(10, 10, 60, 30)) + + def _current() -> AppState: running = reduce(_ready(), PreviewRequested("preview", "preview-request")) return reduce( @@ -545,6 +566,142 @@ def test_replace_dispatch_includes_replace_intent(window, qtbot) -> None: assert services.commands == [ChooseVideoRequested(replace=True)] +def test_edit_regions_switches_the_original_canvas_to_region_mode( + window, qtbot +) -> None: + value, services = window + value.render_state(_ready_with_crop()) + qtbot.mouseClick( + value.inspector.exclusion_controls.edit_button, + Qt.MouseButton.LeftButton, + ) + canvas = value.original_canvas + assert canvas._geometry is not None # noqa: SLF001 + start = canvas._geometry.transform.source_to_widget(PointF(20, 20)) + end = canvas._geometry.transform.source_to_widget(PointF(50, 35)) + + qtbot.mousePress( + canvas, + Qt.MouseButton.LeftButton, + pos=QPoint(round(start.x), round(start.y)), + ) + qtbot.mouseMove(canvas, QPoint(round(end.x), round(end.y))) + qtbot.mouseRelease( + canvas, + Qt.MouseButton.LeftButton, + pos=QPoint(round(end.x), round(end.y)), + ) + + assert services.commands + assert isinstance(services.commands[-1], ExclusionsChanged) + + +def test_source_context_menu_edit_action_keeps_inspector_mode_in_sync( + window, +) -> None: + value, _ = window + value.render_state(_ready_with_crop()) + canvas = value.original_canvas + assert canvas._geometry is not None # noqa: SLF001 + + menu = canvas._build_context_menu( # noqa: SLF001 + canvas._geometry.transform.source_to_widget(PointF(20, 20)) + ) + edit_action = next( + action + for action in menu.actions() + if action.text() == "Edit exclusion regions" + ) + edit_action.trigger() + + assert canvas._exclusion_edit is True # noqa: SLF001 + assert value.inspector.exclusion_controls.edit_button.isChecked() + + +def test_selected_region_readout_shows_its_stored_oriented_rectangle( + window, qtbot +) -> None: + value, _ = window + region = CropSpec(20, 10, 30, 20) + state = replace( + _ready_with_crop(), + parameters=replace(_ready_with_crop().parameters, exclusions=(region,)), + ) + value.render_state(state) + qtbot.mouseClick( + value.inspector.exclusion_controls.edit_button, + Qt.MouseButton.LeftButton, + ) + canvas = value.original_canvas + assert canvas._geometry is not None # noqa: SLF001 + point = canvas._geometry.transform.source_to_widget(PointF(25, 15)) + qtbot.mouseClick( + canvas, + Qt.MouseButton.LeftButton, + pos=QPoint(round(point.x), round(point.y)), + ) + + assert value.inspector.exclusion_controls.selected_readout.text() == ( + "Selected region: x 20, y 10, width 30, height 20 source pixels" + ) + assert ( + value.inspector.exclusion_controls.selected_readout.accessibleDescription() + == value.inspector.exclusion_controls.selected_readout.text() + ) + + +def test_region_readout_reports_no_selection_after_leaving_region_mode( + window, qtbot +) -> None: + value, _ = window + region = CropSpec(20, 10, 30, 20) + state = replace( + _ready_with_crop(), + parameters=replace(_ready_with_crop().parameters, exclusions=(region,)), + ) + value.render_state(state) + controls = value.inspector.exclusion_controls + qtbot.mouseClick(controls.edit_button, Qt.MouseButton.LeftButton) + canvas = value.original_canvas + assert canvas._geometry is not None # noqa: SLF001 + point = canvas._geometry.transform.source_to_widget(PointF(25, 15)) + qtbot.mouseClick( + canvas, + Qt.MouseButton.LeftButton, + pos=QPoint(round(point.x), round(point.y)), + ) + canvas.set_exclusion_edit(False) + + assert controls.selected_readout.text() == "No region selected" + + +def test_overhanging_region_readout_uses_the_cut_exclusions_intersection( + window, qtbot +) -> None: + value, _ = window + region = CropSpec(60, 20, 20, 10) + state = _ready_with_crop() + state = replace(state, parameters=replace(state.parameters, exclusions=(region,))) + value.render_state(state) + controls = value.inspector.exclusion_controls + qtbot.mouseClick(controls.edit_button, Qt.MouseButton.LeftButton) + canvas = value.original_canvas + assert canvas._geometry is not None # noqa: SLF001 + point = canvas._geometry.transform.source_to_widget(PointF(61, 21)) + qtbot.mouseClick( + canvas, + Qt.MouseButton.LeftButton, + pos=QPoint(round(point.x), round(point.y)), + ) + + effective = cut_exclusions((region,), state.crop) + assert effective + assert controls.selected_readout.text() == ( + "Selected region: x 60, y 20, width 20, height 10 source pixels; " + "partly outside crop" + ) + + def test_invalid_saved_geometry_falls_back_to_default(window) -> None: value, _ = window settings = value._settings diff --git a/tests/ui/test_minimum_layout.py b/tests/ui/test_minimum_layout.py index ccff4ea..03d40e3 100644 --- a/tests/ui/test_minimum_layout.py +++ b/tests/ui/test_minimum_layout.py @@ -3,8 +3,11 @@ from collections.abc import Callable from PySide6.QtCore import QSettings +from PySide6.QtGui import QFontMetrics from PySide6.QtWidgets import QApplication +from matteloop.core.parameters import ParameterState +from matteloop.core.specs import CropSpec from matteloop.core.state import AppState, SourceLoaded, SourceLoadRequested, reduce from matteloop.ui.main_window import MainWindow from matteloop.ui.theme import install_theme @@ -99,6 +102,40 @@ def test_inspector_content_minimum_width_fits_each_scroll_viewport(qtbot) -> Non application.setFont(original_font) +def test_exclusion_count_is_fully_visible_at_the_narrowest_inspector_width( + qtbot, +) -> None: + application = QApplication.instance() + assert application is not None + original_style_sheet = application.styleSheet() + original_font = application.font() + try: + install_theme(application) + settings = QSettings( + QSettings.IniFormat, QSettings.UserScope, "matteloop-test", "count-width" + ) + settings.clear() + state = AppState( + parameters=ParameterState(exclusions=(CropSpec(10, 10, 20, 20),)) + ) + window = MainWindow(Store(state), Services(), settings) + qtbot.addWidget(window) + window.resize(1100, 720) + window.show() + for _button, body in window.inspector.disclosures.values(): + body.setVisible(True) + application.processEvents() + + count = window.inspector.exclusion_controls.count_label + text_width = QFontMetrics(count.font()).horizontalAdvance(count.text()) + + assert window.inspector_scroll.viewport().width() == 322 + assert "…" in count.text() or text_width <= count.contentsRect().width() + finally: + application.setStyleSheet(original_style_sheet) + application.setFont(original_font) + + def test_empty_has_only_restrained_source_surface(qtbot) -> None: settings = QSettings( QSettings.IniFormat, QSettings.UserScope, "matteloop-test", "empty" diff --git a/tests/ui/test_parameter_inspector.py b/tests/ui/test_parameter_inspector.py index a542738..40d79c6 100644 --- a/tests/ui/test_parameter_inspector.py +++ b/tests/ui/test_parameter_inspector.py @@ -1,5 +1,6 @@ from __future__ import annotations +from dataclasses import replace from decimal import Decimal from fractions import Fraction from pathlib import Path @@ -8,7 +9,12 @@ from PySide6.QtCore import QSettings, Qt from PySide6.QtWidgets import QAbstractSpinBox, QComboBox -from matteloop.core.parameters import ParametersReset, ParameterState +from matteloop.core.parameters import ( + ExclusionsBeforeModelChanged, + ExclusionsChanged, + ParametersReset, + ParameterState, +) from matteloop.core.specs import CropSpec, EdgeMode, TransformSpec from matteloop.core.state import AppState, SourceState from matteloop.core.timeline import TimelineState @@ -260,6 +266,129 @@ def test_inspector_emits_parameter_commands_from_standard_controls(qtbot) -> Non ] +def test_exclusion_controls_toggle_edit_mode_and_clear_regions(qtbot) -> None: + inspector = Inspector(_settings()) + qtbot.addWidget(inspector) + state = AppState( + source=SourceState.READY, + source_id="source", + source_value=object(), + parameters=ParameterState(exclusions=(CropSpec(10, 10, 20, 20),)), + timeline=TimelineState(Fraction(4), Fraction(0), Fraction(4), Fraction(0)), + ) + store = ReducerStore(state) + controller = SourceController.__new__(SourceController) + controller._store = store + controller._settings = None + controller._closed = False + inspector.apply_parameters(present_parameters(store.state), editable=True) + commands: list[object] = [] + modes: list[bool] = [] + inspector.command_requested.connect(commands.append) + inspector.command_requested.connect(controller.dispatch) + inspector.exclusion_controls.edit_toggled.connect(modes.append) + + controls = inspector.exclusion_controls + assert controls.edit_button.text() == "Edit" + assert controls.edit_button.accessibleName() == "Edit exclusion regions" + assert controls.edit_button.toolTip() == "Edit exclusion regions" + assert controls.clear_button.text() == "Clear" + assert controls.clear_button.accessibleName() == "Clear exclusion regions" + assert controls.count_label.text() == "1 region(s)" + assert controls.clear_button.isEnabled() + + controls.edit_button.click() + controls.clear_button.click() + + assert modes == [True] + assert commands == [ExclusionsChanged(())] + assert store.state.parameters.exclusions == () + inspector.apply_parameters(present_parameters(store.state), editable=False) + + assert modes == [True, False] + assert not controls.edit_button.isEnabled() + assert not controls.clear_button.isEnabled() + assert controls.count_label.text() == "0 region(s)" + + +def test_before_model_checkbox_dispatches_and_is_inert_without_regions(qtbot) -> None: + inspector = Inspector(_settings()) + qtbot.addWidget(inspector) + empty = AppState( + source=SourceState.READY, + source_id="source", + source_value=object(), + timeline=TimelineState(Fraction(4), Fraction(0), Fraction(4), Fraction(0)), + ) + inspector.apply_parameters(present_parameters(empty), editable=True) + commands: list[object] = [] + inspector.command_requested.connect(commands.append) + + checkbox = inspector.exclusion_controls.before_model_checkbox + assert not checkbox.isEnabled() + checkbox.click() + assert commands == [] + + region = CropSpec(10, 10, 20, 20) + with_region = replace( + empty, + parameters=replace(empty.parameters, exclusions=(region,)), + ) + inspector.apply_parameters(present_parameters(with_region), editable=True) + assert checkbox.isEnabled() + checkbox.click() + assert commands == [ExclusionsBeforeModelChanged(True)] + + enabled = replace( + with_region, + parameters=replace( + with_region.parameters, exclusions_before_model=True + ), + ) + inspector.apply_parameters(present_parameters(enabled), editable=True) + assert checkbox.isChecked() + assert inspector.exclusion_controls.count_label.text() == ( + "1 region(s), blanked before the model" + ) + + +def test_before_model_count_label_names_the_blank(qtbot) -> None: + inspector = Inspector(_settings()) + qtbot.addWidget(inspector) + state = AppState( + source=SourceState.READY, + source_id="source", + source_value=object(), + parameters=ParameterState( + exclusions=(CropSpec(10, 10, 20, 20),), + exclusions_before_model=True, + ), + timeline=TimelineState(Fraction(4), Fraction(0), Fraction(4), Fraction(0)), + ) + + inspector.apply_parameters(present_parameters(state), editable=True) + + assert inspector.exclusion_controls.count_label.text() == ( + "1 region(s), blanked before the model" + ) + + +def test_reset_is_enabled_when_exclusion_regions_differ_from_defaults(qtbot) -> None: + inspector = Inspector(_settings()) + qtbot.addWidget(inspector) + state = AppState( + source=SourceState.READY, + source_id="source", + source_value=object(), + parameters=ParameterState(exclusions=(CropSpec(10, 10, 20, 20),)), + timeline=TimelineState(Fraction(4), Fraction(0), Fraction(4), Fraction(0)), + ) + + inspector.apply_parameters(present_parameters(state), editable=True) + + assert inspector.reset_parameters_button.isEnabled() + + def test_inspector_emits_edge_mode_from_the_standard_combo(qtbot) -> None: inspector = Inspector(_settings()) qtbot.addWidget(inspector) diff --git a/tests/ui/test_parameter_persistence.py b/tests/ui/test_parameter_persistence.py index ab173ee..974600c 100644 --- a/tests/ui/test_parameter_persistence.py +++ b/tests/ui/test_parameter_persistence.py @@ -1,9 +1,12 @@ from __future__ import annotations +from dataclasses import dataclass from decimal import Decimal +from fractions import Fraction from pathlib import Path from PySide6.QtCore import QSettings +from PySide6.QtGui import QImage from matteloop.core.execution_providers import ( CPU_EXECUTION_PROVIDER, @@ -16,8 +19,13 @@ ParameterState, ) from matteloop.core.specs import CropSpec, EdgeMode, TransformSpec -from matteloop.core.state import AppState, SourceState -from matteloop.ui.controller import SourceController +from matteloop.core.state import ( + AppState, + SourceLoadRequested, + SourceState, + reduce, +) +from matteloop.ui.controller import LoadedSource, SourceController from matteloop.ui.preferences import load_parameters, persist_parameters from matteloop.ui.store import ReducerStore @@ -33,6 +41,14 @@ def _settings() -> QSettings: return settings +@dataclass(frozen=True) +class _SourceMetadata: + width: int + height: int + duration: Fraction = Fraction(2) + average_rate: Fraction = Fraction(30) + + def test_parameter_preferences_round_trip_as_qsettings_primitives() -> None: settings = _settings() expected = ParameterState( @@ -44,6 +60,7 @@ def test_parameter_preferences_round_trip_as_qsettings_primitives() -> None: alpha_threshold=Decimal("3.5"), padding=9, stretch_x=Decimal("1.2"), + exclusions=(CropSpec(8, 9, 40, 50), CropSpec(80, 20, 10, 15)), output_directory=Path("/exports"), output_filename="last.webp", max_mib=Decimal("2.5"), @@ -61,6 +78,19 @@ def test_parameter_preferences_round_trip_as_qsettings_primitives() -> None: assert actual == expected +def test_exclusions_before_model_persists_as_a_bool() -> None: + settings = _settings() + expected = ParameterState( + exclusions=(CropSpec(8, 9, 40, 50),), + exclusions_before_model=True, + ) + + persist_parameters(settings, expected) + + assert settings.value("parameters/exclusions_before_model") is True + assert load_parameters(settings).exclusions_before_model is True + + def test_persist_parameters_writes_no_transform_key() -> None: settings = _settings() non_identity = ParameterState( @@ -88,6 +118,61 @@ def test_unavailable_saved_provider_falls_back_to_the_available_cpu_choice() -> assert actual.execution_provider == CPU_EXECUTION_PROVIDER +def test_malformed_exclusions_preference_falls_back_to_empty_regions() -> None: + settings = _settings() + settings.setValue("parameters/model_id", "u2net") + settings.setValue("parameters/exclusions", "8,9,40,50;malformed") + + class ReadingSettings: + def __init__(self, source: QSettings) -> None: + self.source = source + self.keys: list[str] = [] + + def value(self, key: str) -> object: + self.keys.append(key) + return self.source.value(key) + + reading = ReadingSettings(settings) + actual = load_parameters(reading) # type: ignore[arg-type] + + assert actual.model_id == "u2net" + assert actual.exclusions == () + assert "parameters/exclusions" in reading.keys + + +def test_source_load_persists_clipped_exclusions_for_the_next_start() -> None: + settings = _settings() + saved = ParameterState( + exclusions=(CropSpec(60, 70, 80, 20), CropSpec(120, 0, 10, 10)) + ) + persist_parameters(settings, saved) + + store = ReducerStore( + reduce( + AppState(parameters=load_parameters(settings)), + SourceLoadRequested("smaller", "load-2"), + ) + ) + controller = SourceController.__new__(SourceController) + controller._store = store + controller._settings = settings + controller._closed = False + + controller._source_loaded( + "smaller", + "load-2", + LoadedSource( + _SourceMetadata(100, 80), + QImage(100, 80, QImage.Format.Format_RGBA8888), + ), + ) + + assert store.state.source is SourceState.READY + assert store.state.parameters.exclusions == (CropSpec(60, 70, 40, 10),) + restarted = AppState(parameters=load_parameters(settings)) + assert restarted.parameters.exclusions == (CropSpec(60, 70, 40, 10),) + + def test_failed_provider_is_not_reintroduced_by_a_later_parameter_save() -> None: settings = _settings() store = ReducerStore( diff --git a/tests/ui/test_parameter_presentation.py b/tests/ui/test_parameter_presentation.py index f7945f2..a014eec 100644 --- a/tests/ui/test_parameter_presentation.py +++ b/tests/ui/test_parameter_presentation.py @@ -1,6 +1,6 @@ from __future__ import annotations -from dataclasses import dataclass +from dataclasses import dataclass, replace from fractions import Fraction from pathlib import Path @@ -15,6 +15,7 @@ SourceLoadRequested, reduce, ) +from matteloop.ui.crop_presentation import present_crop from matteloop.ui.parameter_presentation import present_parameters @@ -63,3 +64,27 @@ def test_parameter_presentation_exposes_the_transform_and_last_artifact() -> Non assert presentation.transform == transform assert presentation.artifact == artifact + + +def test_parameter_and_crop_presentations_carry_exclusion_regions() -> None: + source = Path("/clips/holiday.mp4") + loading = reduce(AppState(), SourceLoadRequested("source", "load")) + ready = reduce(loading, SourceLoaded("source", "load", Metadata(source))) + region = CropSpec(10, 20, 30, 40) + state = replace( + ready, + parameters=replace( + ready.parameters, + exclusions=(region,), + exclusions_before_model=True, + ), + ) + + parameter_presentation = present_parameters(state) + crop_presentation = present_crop(state) + + assert parameter_presentation.exclusions == (region,) + assert parameter_presentation.exclusions_before_model is True + assert crop_presentation is not None + assert crop_presentation.exclusions == (region,) + assert crop_presentation.exclusions_before_model is True diff --git a/tests/ui/test_parameter_requests.py b/tests/ui/test_parameter_requests.py index 21c0351..47569f2 100644 --- a/tests/ui/test_parameter_requests.py +++ b/tests/ui/test_parameter_requests.py @@ -1,6 +1,6 @@ from __future__ import annotations -from dataclasses import dataclass +from dataclasses import dataclass, replace from decimal import Decimal from fractions import Fraction from pathlib import Path @@ -32,6 +32,7 @@ def test_preview_and_render_requests_share_every_inspector_parameter() -> None: alpha_threshold=Decimal("4.5"), padding=6, stretch_x=Decimal("1.25"), + exclusions=(CropSpec(10, 20, 30, 40),), output_directory=Path("/exports"), output_filename="holiday-cut.webp", max_mib=Decimal("12.5"), @@ -48,14 +49,23 @@ def test_preview_and_render_requests_share_every_inspector_parameter() -> None: assert request.segmentation.model_id == parameters.model_id assert request.segmentation.edge_mode is parameters.edge_mode assert request.segmentation.execution_provider == parameters.execution_provider + assert request.segmentation.exclusions == () assert request.framing.trim is parameters.trim assert request.framing.alpha_threshold == parameters.alpha_threshold assert request.framing.padding == parameters.padding assert request.framing.stretch_x == parameters.stretch_x + assert request.framing.exclusions == parameters.exclusions assert request.output.path == Path("/exports/holiday-cut.webp") assert request.output.max_bytes == int(Decimal("12.5") * 1024 * 1024) assert request.transform == transform + enabled = replace(parameters, exclusions_before_model=True) + enabled_request = _render_request( + _preview_inputs(Metadata(source), timeline, parameters=enabled) + ) + assert enabled_request.segmentation.exclusions == enabled.exclusions + assert enabled_request.framing.exclusions == enabled.exclusions + def timeline_to_sampling(timeline: TimelineState): from matteloop.core.specs import SamplingSpec diff --git a/tests/ui/test_result_player.py b/tests/ui/test_result_player.py index 8a31fd9..6c15585 100644 --- a/tests/ui/test_result_player.py +++ b/tests/ui/test_result_player.py @@ -12,6 +12,7 @@ from matteloop.core.geometry import ( _MEDIA_INSET, CropGeometryState, + PixelBounds, RectF, SizeF, build_crop_geometry, @@ -483,6 +484,30 @@ def test_frame_load_worker_reports_unreadable_frames(tmp_path) -> None: assert failures == ["Cut frames could not be read"] +def test_frame_load_worker_masks_exclusion_boxes_before_framing(tmp_path) -> None: + artifact = _seed_cut(tmp_path, "worker-exclusions") + manifest = artifact.manifest + plan = framing_plan((manifest.width, manifest.height), None, FramingSpec()) + worker = FrameLoadWorker( + artifact.cut_workspace, + manifest, + _DirectFrameReader(), + plan, + TransformSpec(), + (100,) * manifest.frame_count, + 1, + exclusions=(PixelBounds(32, 24, 64, 104),), + ) + results: list[PlayerFrames] = [] + worker.succeeded.connect(lambda frames, _generation: results.append(frames)) + + worker.run() + + assert len(results) == 1 + assert results[0].framed[0].pixelColor(40, 30).alpha() == 0 + assert results[0].framed[0].pixelColor(65, 30).alpha() == 255 + + class _CancelAfterFirstReadReader: """Read normally, but signal cancellation once the first frame is read -- simulating a supersede that lands while the worker is mid-load.""" diff --git a/tests/ui/test_transform_stage.py b/tests/ui/test_transform_stage.py index b2e6802..32c5bba 100644 --- a/tests/ui/test_transform_stage.py +++ b/tests/ui/test_transform_stage.py @@ -15,8 +15,10 @@ import matteloop.ui.worker_thread as worker_thread_module from matteloop.core.crop_state import CropChanged +from matteloop.core.geometry import PixelBounds from matteloop.core.parameters import ( AlphaThresholdChanged, + ExclusionsChanged, GlobalTrimChanged, PaddingChanged, TransformChanged, @@ -47,7 +49,13 @@ from matteloop.ui.result_player import PlayerFrames, ResultPlayerCanvas from matteloop.ui.store import ReducerStore from matteloop.ui.transform_group import CutFacts, TransformGroup -from matteloop.ui.transform_stage import TransformStageController, _DirectFrameReader +from matteloop.ui.transform_stage import ( + CutSession, + TransformStageController, + _compute_facts, + _DirectFrameReader, + _resolve_union, +) from tests.jobs.render_support import job, render_service, request @@ -338,6 +346,110 @@ def test_padding_change_recomputes_facts_from_the_shared_framing_stage( controller.shutdown() +def test_facts_ignore_cached_union_metadata_whose_fingerprint_does_not_match( + tmp_path, qtbot +) -> None: + artifact = render_service(workspace=FilesystemWorkspacePort()).render( + request(tmp_path, framing=FramingSpec(True, Decimal("2"), 48, Decimal("1"))), + job(tmp_path, "stale-union", JobKind.RENDER), + ) + framing = FramingSpec( + True, + Decimal("2"), + 48, + Decimal("1"), + exclusions=(CropSpec(32, 24, 32, 80),), + ) + session = CutSession( + artifact.cut_workspace, + artifact.manifest, + artifact.manifest.cache_key_inputs["sampling"]["fps"], + ) + + union = _resolve_union(session, framing, _DirectFrameReader()) + + assert union == PixelBounds(64, 24, 96, 104) + + +def test_empty_union_degrades_to_the_untrimmed_framed_size(tmp_path, qtbot) -> None: + artifact = _seed_cut(tmp_path, "empty-union-facts") + framing = FramingSpec( + True, + Decimal("2"), + 0, + Decimal("1"), + exclusions=(CropSpec(0, 0, 128, 128),), + ) + session = CutSession(artifact.cut_workspace, artifact.manifest, 15) + + facts, plan = _compute_facts(session, framing, _DirectFrameReader()) + + assert plan.global_bounds is None + assert facts.framed_size == (128, 128) + + +def test_region_change_recomputes_facts_and_reloads_player_frames_without_the_region( + tmp_path, qtbot +) -> None: + artifact = _seed_cut(tmp_path, "region-player") + store = ReducerStore(_ready_state(tmp_path / "source.mp4")) + controller = TransformStageController(store) + canvas = ResultPlayerCanvas() + qtbot.addWidget(canvas) + group = TransformGroup(lambda _event: None) + qtbot.addWidget(group) + controller.attach(group, canvas) + controller.open_artifact(artifact) + qtbot.waitUntil(lambda: canvas._frames is not None, timeout=5000) + first_frames = canvas._frames + + store.dispatch(ExclusionsChanged((CropSpec(32, 24, 32, 80),))) + + qtbot.waitUntil( + lambda: canvas._frames is not None + and canvas._frames is not first_frames + and canvas._frames.framed[0].pixelColor(40, 30).alpha() == 0, + timeout=5000, + ) + assert canvas._frames.framed[0].pixelColor(65, 30).alpha() == 255 + controller.shutdown() + + +def test_region_change_outside_the_cut_does_not_recompute_or_reload_frames( + tmp_path, qtbot +) -> None: + artifact = _seed_cut(tmp_path, "region-outside-cut") + initial_region = CropSpec(200, 0, 8, 8) + ready = _ready_state(tmp_path / "source.mp4") + state = replace( + ready, + parameters=replace( + ready.parameters, + exclusions=(initial_region,), + ), + ) + reader = _CountingReader() + store = ReducerStore(state) + controller = TransformStageController(store, frame_reader=reader) + canvas = ResultPlayerCanvas() + qtbot.addWidget(canvas) + group = TransformGroup(lambda _event: None) + qtbot.addWidget(group) + controller.attach(group, canvas) + controller.open_artifact(artifact) + qtbot.waitUntil(lambda: canvas._frames is not None, timeout=5000) + initial_reads = reader.calls + + store.dispatch(ExclusionsChanged((CropSpec(208, 0, 8, 8),))) + qtbot.wait(500) + + assert reader.calls == initial_reads + assert controller._framing.exclusions == ( # noqa: SLF001 + CropSpec(208, 0, 8, 8), + ) + controller.shutdown() + + def test_facts_ready_clamps_a_crop_left_over_from_a_larger_framed_size( tmp_path, qtbot ) -> None: