feat(annotator): cross-device navigation — trackpad pan, pinch zoom, hand tool, touch gestures - #578
Merged
Merged
Conversation
…a mode Adds the pure gesture math to `adapters/viewport.ts` — wheel normalisation over both axes, a zoom factor whose softness tells a mouse notch from a pinch, and a two-finger pinch as a scale about a travelling centroid — and wires it through the React adapter. Plain wheel now pans, both axes, which is what gives a trackpad a pan at all; `ctrl`/`cmd`+wheel zooms, and that one branch serves a macOS pinch, a Windows precision-touchpad pinch and a mouse alike. The hand joins as a mode over the top, `h` through the registry and `Space` held as an adapter substitution, and two touch pointers are a gesture regardless of tool. cf. #576
… the gestures The strip is no longer absent for a viewer: it renders with the hand and the shortcut sheet and nothing else, which retires the exception it used to carry — 'every control on the palette picks a drawing tool' stopped being true the moment one of them was navigation. The sheet keeps its registry-derived rows and gains a written Navigate section, because a two-finger scroll and a pinch have no chord to be read off. Anything that does have one stays in the derived half. cf. #576
…mputed It asserted the held pixel lands on `pinch.centroidX`, which a mutation taking the centroid from before the move satisfies — `dx` comes off the same field, so the expectation slid exactly as far as the answer did. Both midpoints are now computed in the test. cf. #576
…m breaking it Adds `e2e/touch.spec.ts` — six scenarios over Chromium's own touch input, driven through CDP because Playwright has one touch verb and one contact. Existing wheel-zoom scenarios now hold the modifier through `zoomWheel`; a spec that forgot it would still pass its 'the picture moved' assertions while measuring the wrong gesture. Two things came out of mutating the rules rather than writing them: **The gesture lifetime was wrong.** Clearing it when the *last* finger lifts left a pinch inert until the whole hand came off the glass, so lifting one finger and putting another down did nothing. Below two contacts there is no gesture, and two down again is a new one, re-seeded from where the fingers actually are. The survivor needs no swallowing either — `IDLE_ROW` has a `pointer-down` handler and nothing else, so its stray events are silence. **CDP's `touchEnd` lists the points that LEFT**, not the ones remaining, which is the opposite of the protocol's own wording. Following the wording produces a suite that passes and measures nothing: the survivor a scenario then moves is a contact the browser thinks is gone, so every 'nothing moved' assertion holds for the wrong reason. cf. #576
…was inferred from cf. #576
Three in the cycle suite and a second in `annotate.spec.ts` — the scenario asserts it once per frame and a single-site edit caught only the first. `viewport.spec.ts`'s is left alone: that one is the too-narrow viewport, where the whole editor is unmounted. cf. #576
This was referenced Aug 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #576.
What changed
A pan had exactly one spelling — a middle- or secondary-button drag — and a trackpad, a pen and a finger have no second button to offer. On a laptop there was no gesture that moved the picture at all, while the gesture people actually make, a two-finger scroll, zoomed.
Most of the fix is which side of one branch a wheel event falls on:
ctrlKeyormetaKeyThat single branch serves four devices, because
ctrlKeyon a wheel event is how a browser reports a trackpad pinch — macOS and Windows precision touchpads alike, no gesture API involved — andCtrl/Cmd+wheel is the convention for zooming with a mouse.A bare wheel now pans where it used to zoom. That is the deliberate half and the whole of what makes a trackpad workable. Mouse zoom is the modifier, the widget's
−/+, andmod+0.The rest is more spellings of the same two verbs:
Spaceheld is the hand while it is down. It cannot be a registry row — a keystroke is a press and this needs a release — so it is an adapter substitution, the classenterandescapealready belong to, cleared on blur as well as keyup.h, is the persistent one. Not a fifthTool:tool.tsderives the tool from the active class and stores nothing, so the mode is the host's and arrives aspanTool— the arrangement the suggest tool established.pinchBetweenanswers a scale about a travelling centroid — one gesture, because a pinch that also drifts is one thing and answering it as a zoom then a pan makes the picture jump between them.The non-primary pan is untouched and still unconditional. The hand is a second branch beside it, taken on a primary press, and it sits before the read-only select: somebody navigating a batch they may not edit is who most needs to pan.
Arithmetic lives in
adapters/viewport.tsbeside the transform (normalizedWheel,wheelZoomFactor,pinchBetween;zoomAboutandpanBywere already there), so it is unit-tested without a browser.AnnotatorCanvasholds only the branches.The tuned constant
WHEEL_SOFTNESSmoves 400 → 538, derived rather than picked:120 / ln(1.25) ≈ 538makes one wheel notch worth exactly one press of the+button. It needed retuning becausectrlKeycan no longer tell a pinch from a mouse wheel — both zoom now — sowheelZoomFactorsplits on magnitude instead (|delta| >= 40 px): a notch is a large quantised value, a pinch a stream of small continuous ones.The read-only tool strip
The strip used to be absent entirely for a viewer, and the reason held while every control on it picked a drawing tool. The hand is not a drawing tool, and navigating a batch nobody may edit is most of what a viewer does — so the strip now renders carrying the hand and the shortcut sheet, and loses every other button.
DESIGN.mdamended in three places (tool-strip spec, read-only rule, a new navigation-model entry).Two findings from breaking the rules rather than writing them
A green mutation meant the design was wrong, not the test. The touch gesture was cleared when the last finger lifts; mutating that came back green — nothing could see the rule. Tracing why showed the rule was worse than its mutation: lifting one finger left the pinch inert until the whole hand came off the glass, so a finger swap was dead. It now clears below two contacts and re-forms from where the fingers actually are. The survivor needs no swallowing, and that was measured:
IDLE_ROWcarries apointer-downhandler and nothing else, so its stray moves and its stray lift are silence.CDP's
touchEndlists the points that LEFT, not the ones still down — the opposite of the protocol's own wording ("active touch points on the touch device"). Following the wording produces a suite that passes and measures nothing: the survivor a scenario then moves is a contact the browser thinks is gone, so every "nothing moved" assertion holds for the wrong reason. Only the one scenario whose expected outcome is a change could catch it.Also caught by mutation: the flagship pinch-invariant test asserted the held pixel lands on
pinch.centroidX, which the centroid mutation moves too (dxcomes off the same field), so the expectation slid exactly as far as the answer did. Both midpoints are now computed inside the test.11 mutations run, each reddening a named test.
Three corrections to the original brief
Verified against HEAD rather than taken on trust:
MIN_ZOOM = 0.05, andDESIGN.mdargues explicitly against v1's 30% ("an 8K frame does not fit a laptop pane above about 18%"). Bounds unmoved.−/%/+controls are not in the top bar — they areZoomWidget, floating bottom-right on the stage, deliberately moved off the bar. Untouched and still in sync with gesture zoom.core/—core/geometry/tolerance.tsis documented as the only core module allowed to name a zoom, with a grep audit asserting it.adapters/viewport.tsis already DOM-free, React-free and vitest-tested without jsdom.Deliberately not built
Safari's
gesturestart/gesturechangepath. The browser gates are chromium-only, so it would ship unverifiable, and a trackpad pinch in current Safari arrives asctrl+wheel — which the new branch already handles.Found, not fixed
Nothing. No unrelated defect surfaced.
Test plan
bash scripts/check.shrun in stages (the harness kills a command at ~10 minutes), every stage's verdict recorded:check.sh pythonPASSED— 3318 passed, 13 skipped; ruff lint + format, mypy, import-linter cleancheck.sh frontend generated docsPASSED— annotator 1021, ui-core 977, every drift gate, docs buildcheck.sh browserPASSED— annotator e2e 270 passed, browser cycle 1 passedNew coverage: 20 vitest cases in
viewport.test.ts, abindings.test.tsrow forhand the space bar, 5 intoolPalette.test.tsx, and 12 Playwright scenarios — 6 in a newe2e/touch.spec.tsdriving Chromium's own touch input over CDP, plus wheel-pan, sideways pan,Space+drag and the hand tool.Every existing wheel-zoom scenario now holds the modifier through a new
zoomWheelhelper; a spec that forgot it would still pass its "the picture moved" assertions while measuring the wrong gesture.page.mouse.wheeldoes carry a held keyboard modifier — verified, that was the load-bearing unknown.Rebased onto
68ed18a(#577,docs-site/-only, zero file overlap) andcheck.sh frontend generated docsre-run green on the rebased head — that is the stage the docs projection could interact with. The python and browser stages were verified pre-rebase; #577 touches neither surface.What a reviewer should drive by hand
Three paths have no automated coverage on this hardware: two-finger trackpad scroll, trackpad pinch, and the
grab/grabbingcursor. Touch is verified only through Chromium's emulation — no contact area, no palm rejection, none of the jitter a real hand has.