Skip to content

feat: add bounded semantic hover - #210

Merged
yashranaway merged 3 commits into
mainfrom
feat/35-semantic-hover
Sep 22, 2026
Merged

yashranaway merged 3 commits into
mainfrom
feat/35-semantic-hover

Conversation

@SarthakWade

Copy link
Copy Markdown
Collaborator

Summary

  • add hover as a semantic CLI, wire protocol, MCP, and SDK command using ref or role/name targets
  • dispatch trusted pointer movement through Chromium CDP and bounded synthetic pointer/mouse transitions in WKWebView
  • surface login forms revealed by hover through the existing AUTH_REQUIRED challenge flow
  • document engine behavior and track drag separately in Design bounded semantic drag interactions #208

Security

  • preserve the private Unix-socket transport and existing navigation policy
  • reject missing, conflicting, stale, hidden, detached, and obscured targets before dispatch
  • keep page-derived coordinates internal and exclude hover from flow recording
  • validate target unions before transport in the generated TypeScript and Python SDKs

Testing

  • pnpm test
  • pnpm test:runtime
  • pnpm test:npm
  • Python generator, Ruff, strict mypy, and pytest (42 tests)
  • Rust protocol tests and Clippy
  • pnpm build
  • pnpm test:e2e:linux
  • targeted macOS WKWebView hover, navigation, and authentication E2E paths

Closes #35

@SarthakWade SarthakWade added the macos-e2e Run the macOS WKWebView E2E suite label Sep 22, 2026

@yashranaway yashranaway left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thermo-nuclear + deslop: defer.

Per prior flag and PR body, edge + refresh behavior for hover still undefined. If hover reveals auth/forms but refresh invalidates refs without revalidation (like #212 does), we risk stale revealed surface. Also flow recording exclusion needs explicit decision.

Blockers before merge:

  1. Refresh / stale / hidden / obscured revalidation for hover
  2. Obscured-target check before dispatch (dropdowns often obscured)
  3. Document why hover never records in flow vs screenshot series

Code otherwise clean, protocol +2 minimal, no rewrite needed. Re-approve once 1-3 clarified.

SarthakWade and others added 2 commits September 22, 2026 18:56
Share one post-scroll hit test for the synthetic and CDP paths, reject a
covered target before any pointer event, and pause recording when a hover
handler navigates. Record why hover is not replayed in a flow.
@yashranaway
yashranaway force-pushed the feat/35-semantic-hover branch from 34c706a to c64c2ff Compare September 22, 2026 18:57
The hover target checks and the region screenshot checks overflowed one
function once both landed. Split them into helpers.

@yashranaway yashranaway left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hover now resolves a ref from the latest inspection, rejects a hidden, detached, or covered target before any pointer event, and stays out of flow replay. Recording pauses when a hover handler navigates.

@yashranaway
yashranaway merged commit 7c9a1c2 into main Sep 22, 2026
25 of 27 checks passed
@yashranaway
yashranaway deleted the feat/35-semantic-hover branch September 22, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

macos-e2e Run the macOS WKWebView E2E suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

C7 — Design review for deferred command ideas (hover, drag, select, isolation)

2 participants