Skip to content

feat(annotator): cross-device navigation — trackpad pan, pinch zoom, hand tool, touch gestures - #578

Merged
JArmandoAnaya merged 7 commits into
mainfrom
feat/annotator-navigation
Aug 14, 2026
Merged

feat(annotator): cross-device navigation — trackpad pan, pinch zoom, hand tool, touch gestures#578
JArmandoAnaya merged 7 commits into
mainfrom
feat/annotator-navigation

Conversation

@JArmandoAnaya

Copy link
Copy Markdown
Contributor

Closes #576.

What changed

A pan had exactly one spelling — a middle- or secondary-button drag — and a trackpad, a pen and a finger have no second button to offer. On a laptop there was no gesture that moved the picture at all, while the gesture people actually make, a two-finger scroll, zoomed.

Most of the fix is which side of one branch a wheel event falls on:

Held What a wheel event means
ctrlKey or metaKey zoom, anchored at the cursor
nothing pan, both axes

That single branch serves four devices, because ctrlKey on a wheel event is how a browser reports a trackpad pinch — macOS and Windows precision touchpads alike, no gesture API involved — and Ctrl/Cmd+wheel is the convention for zooming with a mouse.

A bare wheel now pans where it used to zoom. That is the deliberate half and the whole of what makes a trackpad workable. Mouse zoom is the modifier, the widget's /+, and mod+0.

The rest is more spellings of the same two verbs:

  • Space held is the hand while it is down. It cannot be a registry row — a keystroke is a press and this needs a release — so it is an adapter substitution, the class enter and escape already belong to, cleared on blur as well as keyup.
  • The hand tool, h, is the persistent one. Not a fifth Tool: tool.ts derives the tool from the active class and stores nothing, so the mode is the host's and arrives as panTool — the arrangement the suggest tool established.
  • Two touch pointers are a gesture whatever tool is armed. pinchBetween answers a scale about a travelling centroid — one gesture, because a pinch that also drifts is one thing and answering it as a zoom then a pan makes the picture jump between them.

The non-primary pan is untouched and still unconditional. The hand is a second branch beside it, taken on a primary press, and it sits before the read-only select: somebody navigating a batch they may not edit is who most needs to pan.

Arithmetic lives in adapters/viewport.ts beside the transform (normalizedWheel, wheelZoomFactor, pinchBetween; zoomAbout and panBy were already there), so it is unit-tested without a browser. AnnotatorCanvas holds only the branches.

The tuned constant

WHEEL_SOFTNESS moves 400 → 538, derived rather than picked: 120 / ln(1.25) ≈ 538 makes one wheel notch worth exactly one press of the + button. It needed retuning because ctrlKey can no longer tell a pinch from a mouse wheel — both zoom now — so wheelZoomFactor splits on magnitude instead (|delta| >= 40 px): a notch is a large quantised value, a pinch a stream of small continuous ones.

The read-only tool strip

The strip used to be absent entirely for a viewer, and the reason held while every control on it picked a drawing tool. The hand is not a drawing tool, and navigating a batch nobody may edit is most of what a viewer does — so the strip now renders carrying the hand and the shortcut sheet, and loses every other button. DESIGN.md amended in three places (tool-strip spec, read-only rule, a new navigation-model entry).

Two findings from breaking the rules rather than writing them

A green mutation meant the design was wrong, not the test. The touch gesture was cleared when the last finger lifts; mutating that came back green — nothing could see the rule. Tracing why showed the rule was worse than its mutation: lifting one finger left the pinch inert until the whole hand came off the glass, so a finger swap was dead. It now clears below two contacts and re-forms from where the fingers actually are. The survivor needs no swallowing, and that was measured: IDLE_ROW carries a pointer-down handler and nothing else, so its stray moves and its stray lift are silence.

CDP's touchEnd lists the points that LEFT, not the ones still down — the opposite of the protocol's own wording ("active touch points on the touch device"). Following the wording produces a suite that passes and measures nothing: the survivor a scenario then moves is a contact the browser thinks is gone, so every "nothing moved" assertion holds for the wrong reason. Only the one scenario whose expected outcome is a change could catch it.

Also caught by mutation: the flagship pinch-invariant test asserted the held pixel lands on pinch.centroidX, which the centroid mutation moves too (dx comes off the same field), so the expectation slid exactly as far as the answer did. Both midpoints are now computed inside the test.

11 mutations run, each reddening a named test.

Three corrections to the original brief

Verified against HEAD rather than taken on trust:

  1. Zoom floor is 5%, not 30%MIN_ZOOM = 0.05, and DESIGN.md argues explicitly against v1's 30% ("an 8K frame does not fit a laptop pane above about 18%"). Bounds unmoved.
  2. The /%/+ controls are not in the top bar — they are ZoomWidget, floating bottom-right on the stage, deliberately moved off the bar. Untouched and still in sync with gesture zoom.
  3. Viewport math does not go in core/core/geometry/tolerance.ts is documented as the only core module allowed to name a zoom, with a grep audit asserting it. adapters/viewport.ts is already DOM-free, React-free and vitest-tested without jsdom.

Deliberately not built

Safari's gesturestart/gesturechange path. The browser gates are chromium-only, so it would ship unverifiable, and a trackpad pinch in current Safari arrives as ctrl+wheel — which the new branch already handles.

Found, not fixed

Nothing. No unrelated defect surfaced.

Test plan

bash scripts/check.sh run in stages (the harness kills a command at ~10 minutes), every stage's verdict recorded:

Stage Verdict
check.sh python PASSED — 3318 passed, 13 skipped; ruff lint + format, mypy, import-linter clean
check.sh frontend generated docs PASSED — annotator 1021, ui-core 977, every drift gate, docs build
check.sh browser PASSED — annotator e2e 270 passed, browser cycle 1 passed

New coverage: 20 vitest cases in viewport.test.ts, a bindings.test.ts row for h and the space bar, 5 in toolPalette.test.tsx, and 12 Playwright scenarios — 6 in a new e2e/touch.spec.ts driving Chromium's own touch input over CDP, plus wheel-pan, sideways pan, Space+drag and the hand tool.

Every existing wheel-zoom scenario now holds the modifier through a new zoomWheel helper; a spec that forgot it would still pass its "the picture moved" assertions while measuring the wrong gesture. page.mouse.wheel does carry a held keyboard modifier — verified, that was the load-bearing unknown.

Rebased onto 68ed18a (#577, docs-site/-only, zero file overlap) and check.sh frontend generated docs re-run green on the rebased head — that is the stage the docs projection could interact with. The python and browser stages were verified pre-rebase; #577 touches neither surface.

What a reviewer should drive by hand

Three paths have no automated coverage on this hardware: two-finger trackpad scroll, trackpad pinch, and the grab/grabbing cursor. Touch is verified only through Chromium's emulation — no contact area, no palm rejection, none of the jitter a real hand has.

…a mode

Adds the pure gesture math to `adapters/viewport.ts` — wheel normalisation
over both axes, a zoom factor whose softness tells a mouse notch from a
pinch, and a two-finger pinch as a scale about a travelling centroid — and
wires it through the React adapter.

Plain wheel now pans, both axes, which is what gives a trackpad a pan at
all; `ctrl`/`cmd`+wheel zooms, and that one branch serves a macOS pinch, a
Windows precision-touchpad pinch and a mouse alike. The hand joins as a
mode over the top, `h` through the registry and `Space` held as an adapter
substitution, and two touch pointers are a gesture regardless of tool.

cf. #576
… the gestures

The strip is no longer absent for a viewer: it renders with the hand and
the shortcut sheet and nothing else, which retires the exception it used to
carry — 'every control on the palette picks a drawing tool' stopped being
true the moment one of them was navigation.

The sheet keeps its registry-derived rows and gains a written Navigate
section, because a two-finger scroll and a pinch have no chord to be read
off. Anything that does have one stays in the derived half.

cf. #576
…mputed

It asserted the held pixel lands on `pinch.centroidX`, which a mutation
taking the centroid from before the move satisfies — `dx` comes off the same
field, so the expectation slid exactly as far as the answer did. Both
midpoints are now computed in the test.

cf. #576
…m breaking it

Adds `e2e/touch.spec.ts` — six scenarios over Chromium's own touch input,
driven through CDP because Playwright has one touch verb and one contact.
Existing wheel-zoom scenarios now hold the modifier through `zoomWheel`; a
spec that forgot it would still pass its 'the picture moved' assertions
while measuring the wrong gesture.

Two things came out of mutating the rules rather than writing them:

**The gesture lifetime was wrong.** Clearing it when the *last* finger lifts
left a pinch inert until the whole hand came off the glass, so lifting one
finger and putting another down did nothing. Below two contacts there is no
gesture, and two down again is a new one, re-seeded from where the fingers
actually are. The survivor needs no swallowing either — `IDLE_ROW` has a
`pointer-down` handler and nothing else, so its stray events are silence.

**CDP's `touchEnd` lists the points that LEFT**, not the ones remaining,
which is the opposite of the protocol's own wording. Following the wording
produces a suite that passes and measures nothing: the survivor a scenario
then moves is a contact the browser thinks is gone, so every 'nothing moved'
assertion holds for the wrong reason.

cf. #576
Three in the cycle suite and a second in `annotate.spec.ts` — the scenario
asserts it once per frame and a single-site edit caught only the first.
`viewport.spec.ts`'s is left alone: that one is the too-narrow viewport,
where the whole editor is unmounted.

cf. #576
@JArmandoAnaya
JArmandoAnaya merged commit 69dbcc8 into main Aug 14, 2026
15 checks passed
@JArmandoAnaya
JArmandoAnaya deleted the feat/annotator-navigation branch August 14, 2026 01:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

annotator: cross-device navigation — trackpad pan, pinch zoom, hand tool, touch gestures

1 participant