From 0eb8d7c3843c7148e34cbf754891facb5c5ea072 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 15 Oct 2026 02:52:28 +0000 Subject: [PATCH 1/8] Decide camera, microphone and screen-capture requests (Canvas 35) A host app can now answer a page's media-capture requests: setPermissionHandler(WebViewPermissionHandler) receives origin, url and the kinds asked for (camera, microphone, display) on the EDT and answers allow, deny or default from any thread, now or later. Native hooks: WebKitGTK's permission-request (plus enable-media-stream/enable-webrtc where the build has them), WebView2's PermissionRequested with a deferral, and WKWebView's requestMediaCapturePermissionForOrigin on macOS 12+. With no handler each engine behaves as before. Linux is tested in lightweight mode with mock capture devices; Windows and macOS are compiled by CI but untested. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01WxLMiE3sgxfvWqy44xRLDY --- README.md | 66 +++ demos/WebViewMediaPermissionDemo/README.md | 45 ++ .../demos/WebViewMediaPermissionDemo.java | 143 +++++ demos/WebViewMediaSmokeTest/README.md | 18 + .../WebViewMediaPermissionSmokeTest.java | 228 ++++++++ docs/index.html | 4 + ...-microphone-and-screen-capture-requests.md | 159 ++++++ run-linux-media-permission-demo.sh | 181 +++++++ run-linux-media-smoketest.sh | 41 +- run-mac-media-permission-demo.sh | 125 +++++ run-windows-media-permission-demo.bat | 215 ++++++++ ...-microphone-and-screen-capture-requests.md | 141 +++++ ...-Microphone-And-Screen-Capture-Requests.md | 372 +++++++++++++ src/ca/weblite/webview/EmbeddedWebView.java | 25 + src/ca/weblite/webview/MediaCaptureKind.java | 48 ++ src/ca/weblite/webview/OffscreenWebView.java | 25 + .../weblite/webview/PermissionDecision.java | 29 + .../weblite/webview/PermissionDispatcher.java | 172 ++++++ src/ca/weblite/webview/WebViewNative.java | 15 + .../webview/WebViewPermissionCallback.java | 28 + .../webview/WebViewPermissionHandler.java | 53 ++ .../webview/WebViewPermissionRequest.java | 103 ++++ .../webview/swing/WebViewComponent.java | 50 ++ .../swing/WebViewHeavyweightComponent.java | 13 + .../swing/WebViewLightweightComponent.java | 13 + src_c/webkit_loader.h | 20 +- src_c/webkit_shim.h | 12 + src_c/webview_embed.cpp | 502 ++++++++++++++++++ .../webview/PermissionDispatcherTest.java | 228 ++++++++ windows/webview_embed.cc | 255 +++++++++ 30 files changed, 3318 insertions(+), 11 deletions(-) create mode 100644 demos/WebViewMediaPermissionDemo/README.md create mode 100644 demos/WebViewMediaPermissionDemo/src/ca/weblite/webview/demos/WebViewMediaPermissionDemo.java create mode 100644 demos/WebViewMediaSmokeTest/src/ca/weblite/webview/demos/WebViewMediaPermissionSmokeTest.java create mode 100644 requirements/[User-story-11]decide-camera-microphone-and-screen-capture-requests.md create mode 100755 run-linux-media-permission-demo.sh create mode 100755 run-mac-media-permission-demo.sh create mode 100644 run-windows-media-permission-demo.bat create mode 100644 spdd/analysis/GGQPA-XXX-202610150300-[Analysis]-decide-camera-microphone-and-screen-capture-requests.md create mode 100644 spdd/prompt/35-20261015-0310-[Feat]-Decide-Camera-Microphone-And-Screen-Capture-Requests.md create mode 100644 src/ca/weblite/webview/MediaCaptureKind.java create mode 100644 src/ca/weblite/webview/PermissionDecision.java create mode 100644 src/ca/weblite/webview/PermissionDispatcher.java create mode 100644 src/ca/weblite/webview/WebViewPermissionCallback.java create mode 100644 src/ca/weblite/webview/WebViewPermissionHandler.java create mode 100644 src/ca/weblite/webview/WebViewPermissionRequest.java create mode 100644 test/ca/weblite/webview/PermissionDispatcherTest.java diff --git a/README.md b/README.md index 7fd8959..745ea40 100644 --- a/README.md +++ b/README.md @@ -1006,6 +1006,69 @@ wv.setNavigationHandler(e -> popups of a page you restrict. See [`demos/WebViewNavigationDemo/`](demos/WebViewNavigationDemo/README.md). +## Camera, microphone and screen capture + +A page's `getUserMedia` / `getDisplayMedia` request is put to a permission +handler, which answers **allow**, **deny** or **default** (let the engine +decide) — at once, or later from any thread, for example after asking the +user: + +```java +wv.setPermissionHandler(req -> { // runs on the EDT + if (!req.origin().equals("demo://app")) { req.deny(); return; } + int r = JOptionPane.showConfirmDialog(frame, + req.origin() + " wants " + req.kinds(), "Permission", JOptionPane.YES_NO_OPTION); + if (r == JOptionPane.YES_OPTION) req.allow(); else req.deny(); +}); +``` + +* **The request** gives `origin()` (`scheme://host[:port]`), `url()` and + `kinds()` — a set of `MediaCaptureKind.CAMERA`, `MICROPHONE` and `DISPLAY`. + Answer with `allow()`, `deny()`, `useDefault()` or `answer(PermissionDecision)`; + only the first answer counts (each returns whether it was the first). +* **Threading.** The handler runs on the Swing EDT, delivered with + `invokeLater`; no engine thread waits for it, so a modal dialog is fine. + The page's promise stays pending until the answer arrives. A handler that + throws denies; a request still unanswered when the component is disposed + is denied. +* **No handler, no change.** Without a handler (the default, or + `setPermissionHandler(null)`), and for `useDefault()`, each engine does what + it did before: + + | Engine | Default | + |---|---| + | Linux (WebKitGTK) | refuses (`NotAllowedError`) | + | Windows (WebView2) | WebView2's own permission prompt | + | macOS (WKWebView) | WebKit's own permission prompt | + +* **Platform coverage.** `WebViewComponent.isPermissionHandlerSupported()` + says whether the loaded native routes requests. + * **Linux** (lightweight): camera and microphone, and screen capture with + WebKitGTK 2.42+, through `permission-request`. Every view has + `enable-media-stream` and `enable-webrtc` switched on where the WebKitGTK + version has them. The origin is the top page's (WebKitGTK reports no + frame origin). + * **Windows**: camera and microphone through `PermissionRequested`. Screen + capture has no permission kind in the WebView2 SDK this library builds + against, so WebView2's own picker decides it. + * **macOS 12+**: camera and microphone through the UI delegate's + `requestMediaCapturePermissionForOrigin…`; screen capture keeps WebKit's + own behaviour. Before macOS 12, nothing changes. +* **macOS Info.plist.** The host application must declare + `NSCameraUsageDescription` and `NSMicrophoneUsageDescription`; without them + macOS terminates the process the first time a page uses the device. macOS's + own one-time consent prompt still appears on top of your handler's answer. +* **WebRTC.** Whether `RTCPeerConnection` exists depends on how the engine was + built — Ubuntu 24.04's WebKitGTK has none, even with `enable-webrtc` on. A + page detects it with `typeof RTCPeerConnection === 'function'` and falls + back (for example, to opening the page in the user's browser). +* **Media capture needs a secure context**: `https:`, `localhost`, or a custom + scheme registered with `WebViewSchemes`. +* **Testing on Linux without a camera.** `WEBVIEW_MOCK_CAPTURE_DEVICES=1` + turns on WebKitGTK's mock camera, microphone and screen (a test-only switch). + `xvfb-run -a ./run-linux-media-smoketest.sh` runs the pass/fail check. See + [`demos/WebViewMediaPermissionDemo/`](demos/WebViewMediaPermissionDemo/README.md). + ## Demo See [`demos/WebViewHeavyweightDemo/`](demos/WebViewHeavyweightDemo/README.md) @@ -1048,6 +1111,9 @@ Additional demos: * `demos/WebViewNavigationDemo/` — a page that tries every way to navigate away to a local listener, each refused by a navigation handler before anything is sent (`run-*-navigation-demo`). +* `demos/WebViewMediaPermissionDemo/` — camera, microphone and screen + sharing decided by a permission handler that asks you in a Swing dialog + (`run-*-media-permission-demo`). ## Building from source diff --git a/demos/WebViewMediaPermissionDemo/README.md b/demos/WebViewMediaPermissionDemo/README.md new file mode 100644 index 0000000..b9a46f7 --- /dev/null +++ b/demos/WebViewMediaPermissionDemo/README.md @@ -0,0 +1,45 @@ +# WebViewMediaPermissionDemo + +Camera, microphone and screen-capture requests decided by the application (Canvas 35). + +```bash +./run-linux-media-permission-demo.sh # Linux (lightweight) +MOCK=1 ./run-linux-media-permission-demo.sh # Linux, WebKitGTK's mock devices (no camera needed) +./run-mac-media-permission-demo.sh # macOS 12+ (heavyweight) +run-windows-media-permission-demo.bat # Windows (heavyweight, WebView2) +``` + +The page is served from `demo://app/`, a registered custom scheme, so it is a secure context (media +capture needs one). Its buttons call `getUserMedia` for the camera, the microphone or both, and +`getDisplayMedia` to share the screen. It shows the stream, or the error name on refusal, and whether +the engine has WebRTC (`typeof RTCPeerConnection === 'function'`). + +The **Policy** box chooses what the `WebViewPermissionHandler` does: + +| Policy | Result | +|---|---| +| Ask me (dialog) | a Swing dialog on the EDT asks you; the request is answered when you choose | +| Always allow / Always deny | answered at once | +| Engine default | `useDefault()`: Linux refuses; Windows and macOS show the engine's own prompt | +| No handler | `setPermissionHandler(null)`: the request never reaches the application | + +The log shows each request's origin, kinds and address, and the thread the handler ran on (the EDT). + +## What to check + +1. **Allow** gives a live stream; **Deny** gives `NotAllowedError`. +2. Leaving the dialog open doesn't freeze the page or the window; the page's promise waits. +3. **Engine default** and **No handler** behave the same. +4. On Linux, **Share screen** reaches the handler as `[DISPLAY]` with WebKitGTK 2.42+. On Windows and + macOS it does not reach the handler: WebView2's and WebKit's own pickers decide it. + +## Platform notes + +- **macOS** needs 12 or newer. The process that uses the camera must declare + `NSCameraUsageDescription` and `NSMicrophoneUsageDescription` in its Info.plist, or macOS terminates + it on first access. This script runs a plain `java`, so macOS attributes the request to your terminal + or JDK; a packaged application must declare both keys itself. macOS's own one-time consent prompt + still appears on top of the handler's answer. +- **Windows**: camera and microphone only. Windows' own privacy settings still apply. +- **Linux**: the lightweight component is the only Linux mode. Ubuntu 24.04's WebKitGTK is built + without WebRTC, so `RTCPeerConnection` is missing there even though capture works. diff --git a/demos/WebViewMediaPermissionDemo/src/ca/weblite/webview/demos/WebViewMediaPermissionDemo.java b/demos/WebViewMediaPermissionDemo/src/ca/weblite/webview/demos/WebViewMediaPermissionDemo.java new file mode 100644 index 0000000..0f74ad1 --- /dev/null +++ b/demos/WebViewMediaPermissionDemo/src/ca/weblite/webview/demos/WebViewMediaPermissionDemo.java @@ -0,0 +1,143 @@ +/* + * MIT License + * + * Demo for camera / microphone / screen-capture permission requests + * (Canvas 35 D10). Serves demo://app/ from a custom scheme (a secure + * context). Each button on the page asks for devices; the application's + * WebViewPermissionHandler answers, by default after asking you in a Swing + * dialog. The page shows the stream, or the error name on refusal, and + * whether this engine has WebRTC (RTCPeerConnection). + */ +package ca.weblite.webview.demos; + +import ca.weblite.webview.PermissionDecision; +import ca.weblite.webview.WebViewPermissionHandler; +import ca.weblite.webview.WebViewPermissionRequest; +import ca.weblite.webview.WebViewSchemeHandler; +import ca.weblite.webview.WebViewSchemeResponse; +import ca.weblite.webview.WebViewSchemes; +import ca.weblite.webview.swing.WebViewComponent; + +import java.awt.BorderLayout; +import java.awt.Dimension; +import java.awt.EventQueue; +import java.nio.charset.StandardCharsets; +import java.text.SimpleDateFormat; +import java.util.Date; +import javax.swing.JComboBox; +import javax.swing.JFrame; +import javax.swing.JLabel; +import javax.swing.JOptionPane; +import javax.swing.JPanel; +import javax.swing.JScrollPane; +import javax.swing.JSplitPane; +import javax.swing.JTextArea; + +public class WebViewMediaPermissionDemo { + + private static final String PAGE = "" + + "" + + "
Secure context: · WebRTC (RTCPeerConnection):
" + + "
" + + "" + + "" + + "" + + "" + + "
" + + "
Press a button.
" + + ""; + + private static JTextArea log; + + public static void main(String[] args) throws Exception { + WebViewSchemes.register("demo", new WebViewSchemeHandler() { + @Override + public void handle(ca.weblite.webview.WebViewSchemeRequest request, + ca.weblite.webview.WebViewSchemeResponder responder) { + responder.respond(WebViewSchemeResponse.ok("text/html; charset=utf-8", + PAGE.getBytes(StandardCharsets.UTF_8))); + } + }); + EventQueue.invokeLater(WebViewMediaPermissionDemo::show); + } + + private static void show() { + final JFrame frame = new JFrame("WebViewMediaPermissionDemo"); + frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE); + final WebViewComponent wv = WebViewComponent.create(); + wv.setPreferredSize(new Dimension(760, 420)); + + log = new JTextArea(10, 80); + log.setEditable(false); + + final JComboBox policy = new JComboBox(new String[] { + "Ask me (dialog)", "Always allow", "Always deny", "Engine default (useDefault)", + "No handler (engine decides)"}); + final WebViewPermissionHandler handler = new WebViewPermissionHandler() { + @Override + public void permissionRequested(final WebViewPermissionRequest req) { + log("asked: origin=" + req.origin() + " kinds=" + req.kinds() + " url=" + req.url() + + " (thread " + Thread.currentThread().getName() + ")"); + switch (policy.getSelectedIndex()) { + case 1: answer(req, PermissionDecision.ALLOW); return; + case 2: answer(req, PermissionDecision.DENY); return; + case 3: answer(req, PermissionDecision.DEFAULT); return; + default: break; + } + // Runs on the EDT, and nothing waits for it: a modal dialog is fine. + Object[] options = {"Allow", "Deny", "Engine default"}; + int r = JOptionPane.showOptionDialog(frame, + req.origin() + " wants to use: " + req.kinds(), + "Permission request", JOptionPane.DEFAULT_OPTION, + JOptionPane.QUESTION_MESSAGE, null, options, options[0]); + answer(req, r == 0 ? PermissionDecision.ALLOW + : r == 2 ? PermissionDecision.DEFAULT : PermissionDecision.DENY); + } + }; + policy.addActionListener(e -> { + wv.setPermissionHandler(policy.getSelectedIndex() == 4 ? null : handler); + log("policy: " + policy.getSelectedItem()); + }); + wv.setPermissionHandler(handler); + + JPanel top = new JPanel(new BorderLayout()); + top.add(new JLabel(" Policy: "), BorderLayout.WEST); + top.add(policy, BorderLayout.CENTER); + JPanel north = new JPanel(new BorderLayout()); + north.add(top, BorderLayout.NORTH); + north.add(wv, BorderLayout.CENTER); + JSplitPane split = new JSplitPane(JSplitPane.VERTICAL_SPLIT, north, new JScrollPane(log)); + split.setResizeWeight(0.75); + frame.getContentPane().add(split); + frame.pack(); + frame.setLocationRelativeTo(null); + frame.setVisible(true); + wv.setUrl("demo://app/"); + log("mode=" + WebViewComponent.resolveDefaultMode() + + " isPermissionHandlerSupported=" + WebViewComponent.isPermissionHandlerSupported()); + } + + private static void answer(WebViewPermissionRequest req, PermissionDecision d) { + boolean first = req.answer(d); + log("answered " + d + (first ? "" : " (ignored: already answered)")); + } + + private static void log(final String s) { + final String line = new SimpleDateFormat("HH:mm:ss.SSS").format(new Date()) + " " + s + "\n"; + EventQueue.invokeLater(() -> log.append(line)); + } +} diff --git a/demos/WebViewMediaSmokeTest/README.md b/demos/WebViewMediaSmokeTest/README.md index a5c7139..b727249 100644 --- a/demos/WebViewMediaSmokeTest/README.md +++ b/demos/WebViewMediaSmokeTest/README.md @@ -25,3 +25,21 @@ error); P passed. Needs the GStreamer VP8 decoder (`gstreamer1.0-plugins-good`, installed with WebKitGTK on most distributions). + +## Camera, microphone and screen-capture permissions (Canvas 35) + +The same script then runs `WebViewMediaPermissionSmokeTest`, with `WEBVIEW_MOCK_CAPTURE_DEVICES=1` so +WebKitGTK's mock camera, microphone and screen stand in for real devices. Each phase opens a fresh +lightweight component on `permtest://app/` (a secure context) and asks for devices: + +| Phase | Handler | Page asks | Expected | +|---|---|---|---| +| **1** deny | `deny()` | `getUserMedia({video})` | `NotAllowedError`; handler saw `[CAMERA]` from origin `permtest://app` | +| **2** allow | `allow()` | `getUserMedia({video, audio})` | a stream with `audio,video` tracks; handler saw `[CAMERA, MICROPHONE]` | +| **3** later | `allow()` 1.5 s later from another thread | `getUserMedia({audio})` | an `audio` stream; the EDT stays responsive meanwhile | +| **4** no handler | — | `getUserMedia({video})` | `NotAllowedError` (WebKitGTK's own default) | +| **5** default | `useDefault()` | `getUserMedia({video})` | `NotAllowedError` | +| **6** display | `allow()` | `getDisplayMedia({video})` | handler saw `[DISPLAY]` (SKIP, not FAIL, when no request arrives: WebKitGTK < 2.42) | +| **7** WebRTC | — | `typeof RTCPeerConnection` | informational (`undefined` on Ubuntu 24.04's WebKitGTK 2.52) | + +The script exits `0` only when both smoke tests pass. diff --git a/demos/WebViewMediaSmokeTest/src/ca/weblite/webview/demos/WebViewMediaPermissionSmokeTest.java b/demos/WebViewMediaSmokeTest/src/ca/weblite/webview/demos/WebViewMediaPermissionSmokeTest.java new file mode 100644 index 0000000..bb61c9c --- /dev/null +++ b/demos/WebViewMediaSmokeTest/src/ca/weblite/webview/demos/WebViewMediaPermissionSmokeTest.java @@ -0,0 +1,228 @@ +/* + * MIT License + * + * Pass/fail smoke test for camera / microphone / screen-capture permission + * requests in the Linux lightweight engine (Canvas 35 D11). Exits 0 when + * every check passes, 1 otherwise, printing one line per check, so it can run + * unattended under Xvfb. Needs WebKitGTK's mock capture devices: + * run-linux-media-smoketest.sh exports WEBVIEW_MOCK_CAPTURE_DEVICES=1. + * + * Phases, each in a fresh lightweight component: + * deny handler denies {video} -> NotAllowedError + * allow handler allows {video,audio} -> audio+video tracks + * later allows 1.5 s later {audio} -> audio track; EDT free + * none no handler {video} -> NotAllowedError + * default useDefault() {video} -> NotAllowedError + * display handler allows getDisplayMedia -> handler saw DISPLAY + * (informational: SKIP when no request reaches the handler) + * webrtc typeof RTCPeerConnection (informational) + */ +package ca.weblite.webview.demos; + +import ca.weblite.webview.MediaCaptureKind; +import ca.weblite.webview.WebViewPermissionHandler; +import ca.weblite.webview.WebViewPermissionRequest; +import ca.weblite.webview.WebViewSchemeHandler; +import ca.weblite.webview.WebViewSchemeResponse; +import ca.weblite.webview.WebViewSchemes; +import ca.weblite.webview.swing.WebViewComponent; + +import java.awt.BorderLayout; +import java.awt.Dimension; +import java.awt.EventQueue; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Collections; +import java.util.EnumSet; +import java.util.List; +import java.util.Set; +import java.util.concurrent.TimeUnit; +import javax.swing.JFrame; + +public class WebViewMediaPermissionSmokeTest { + + private static final String PAGE = "" + + "" + + ""; + + /** What the handler was asked. */ + private static final List seen = + Collections.synchronizedList(new ArrayList()); + + private static JFrame frame; + private static boolean ok = true; + + public static void main(String[] args) throws Exception { + WebViewSchemes.register("permtest", new WebViewSchemeHandler() { + @Override + public void handle(ca.weblite.webview.WebViewSchemeRequest request, + ca.weblite.webview.WebViewSchemeResponder responder) { + responder.respond(WebViewSchemeResponse.ok("text/html; charset=utf-8", + PAGE.getBytes(StandardCharsets.UTF_8))); + } + }); + EventQueue.invokeAndWait(() -> { + frame = new JFrame("WebViewMediaPermissionSmokeTest"); + frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE); + frame.getContentPane().setLayout(new BorderLayout()); + frame.setSize(new Dimension(360, 240)); + frame.setVisible(true); + }); + System.err.println("[perm-smoke] supported=" + + WebViewComponent.isPermissionHandlerSupported() + + " WEBVIEW_MOCK_CAPTURE_DEVICES=" + System.getenv("WEBVIEW_MOCK_CAPTURE_DEVICES")); + ok &= check("S isPermissionHandlerSupported", String.valueOf( + WebViewComponent.isPermissionHandlerSupported()), "true"); + + // 1. deny + String r = phase(req -> req.deny(), "{video:true}", false, 15); + ok &= check("1 deny -> NotAllowedError", r, "err:NotAllowedError"); + ok &= check("1 deny: handler kinds", kindsSeen(), EnumSet.of(MediaCaptureKind.CAMERA).toString()); + ok &= check("1 deny: origin", originSeen(), "permtest://app"); + + // 2. allow + r = phase(req -> req.allow(), "{video:true,audio:true}", false, 15); + ok &= check("2 allow -> audio+video stream", r, "ok:audio,video"); + ok &= check("2 allow: handler kinds", kindsSeen(), + EnumSet.of(MediaCaptureKind.CAMERA, MediaCaptureKind.MICROPHONE).toString()); + + // 3. later, from another thread, with the EDT free while waiting + final long[] edtMax = new long[1]; + r = phase(req -> new Thread(() -> { + try { + Thread.sleep(1500); + } catch (InterruptedException ignored) { + // fall through and answer + } + req.allow(); + }, "late-answer").start(), "{audio:true}", false, 15, edtMax); + ok &= check("3 later -> audio stream", r, "ok:audio"); + ok &= check("3 later: EDT responsive while pending (max invokeAndWait ms < 500)", + String.valueOf(edtMax[0]), edtMax[0] < 500 ? null : "< 500"); + + // 4. no handler -> WebKitGTK's own default (refuse) + r = phase(null, "{video:true}", false, 15); + ok &= check("4 no handler -> NotAllowedError", r, "err:NotAllowedError"); + ok &= check("4 no handler: nothing asked", String.valueOf(seen.size()), "0"); + + // 5. useDefault() -> same as no handler + r = phase(req -> req.useDefault(), "{video:true}", false, 15); + ok &= check("5 default -> NotAllowedError", r, "err:NotAllowedError"); + + // 6. display (informational) + r = phase(req -> req.allow(), "{video:true}", true, 8); + if (seen.isEmpty()) { + System.err.println("[perm-smoke] SKIP 6 display: no request reached the handler (result " + + r + "); WebKitGTK < 2.42 or no display-capture backend here"); + } else { + ok &= check("6 display: handler kinds", kindsSeen(), EnumSet.of(MediaCaptureKind.DISPLAY).toString()); + System.err.println("[perm-smoke] INFO 6 display result: " + r); + } + + // 7. WebRTC availability (informational) + System.err.println("[perm-smoke] INFO 7 typeof RTCPeerConnection: " + lastRtc); + + System.err.println("[perm-smoke] " + (ok ? "PASSED" : "FAILED")); + System.exit(ok ? 0 : 1); + } + + private static String lastRtc = "(unknown)"; + + private static String phase(WebViewPermissionHandler h, String constraints, boolean display, + int timeoutSec) throws Exception { + return phase(h, constraints, display, timeoutSec, null); + } + + /** Show the page in a fresh lightweight component with {@code h}, ask, and wait for the outcome. */ + private static String phase(final WebViewPermissionHandler h, String constraints, boolean display, + int timeoutSec, long[] edtMax) throws Exception { + seen.clear(); + final WebViewComponent[] holder = new WebViewComponent[1]; + EventQueue.invokeAndWait(() -> { + frame.getContentPane().removeAll(); + WebViewComponent wv = WebViewComponent.create(WebViewComponent.Mode.LIGHTWEIGHT); + if (h != null) { + wv.setPermissionHandler(req -> { + seen.add(req); + h.permissionRequested(req); + }); + } + frame.getContentPane().add(wv, BorderLayout.CENTER); + frame.getContentPane().revalidate(); + wv.setUrl("permtest://app/"); + holder[0] = wv; + }); + WebViewComponent wv = holder[0]; + long deadline = System.currentTimeMillis() + 20_000; + while (System.currentTimeMillis() < deadline) { + String s = eval(wv, "return typeof window.ask;"); + if ("\"function\"".equals(s)) break; + Thread.sleep(200); + } + lastRtc = unquote(eval(wv, "return typeof RTCPeerConnection;")); + wv.eval("window.ask(" + constraints + ", " + display + ");"); + String result = "(none)"; + deadline = System.currentTimeMillis() + timeoutSec * 1000L; + while (System.currentTimeMillis() < deadline) { + if (edtMax != null) { + long t0 = System.nanoTime(); + EventQueue.invokeAndWait(() -> { }); + edtMax[0] = Math.max(edtMax[0], (System.nanoTime() - t0) / 1_000_000L); + } + String s = unquote(eval(wv, "return window.__result;")); + if (s != null && !s.equals("pending") && !s.equals("null") && !s.isEmpty()) { + result = s; + break; + } + result = s; + Thread.sleep(200); + } + return result; + } + + private static String eval(WebViewComponent wv, String js) { + try { + return wv.evalAsync(js).get(3, TimeUnit.SECONDS); + } catch (Exception e) { + return null; + } + } + + private static String unquote(String s) { + if (s == null) return null; + if (s.length() >= 2 && s.startsWith("\"") && s.endsWith("\"")) return s.substring(1, s.length() - 1); + return s; + } + + private static String kindsSeen() { + if (seen.isEmpty()) return "(no request)"; + Set k = seen.get(0).kinds(); + return EnumSet.copyOf(k).toString(); + } + + private static String originSeen() { + return seen.isEmpty() ? "(no request)" : seen.get(0).origin(); + } + + /** One check line; {@code expected} null means the caller already decided it passed. */ + private static boolean check(String name, String actual, String expected) { + boolean pass = expected == null || expected.equals(actual); + System.err.println("[perm-smoke] " + (pass ? "PASS" : "FAIL") + " " + name + ": " + actual + + (pass ? "" : " (expected " + expected + ")")); + return pass; + } +} diff --git a/docs/index.html b/docs/index.html index f06f127..8cb34d1 100644 --- a/docs/index.html +++ b/docs/index.html @@ -302,6 +302,10 @@

Context menus

Console (standalone)

The non-Swing WebView primitive, driven from a console app.

+ +

Camera & microphone

+

setPermissionHandler: allow, deny or defer to the engine when a page asks for the camera, microphone or screen — answered whenever your UI is ready.

+ diff --git a/requirements/[User-story-11]decide-camera-microphone-and-screen-capture-requests.md b/requirements/[User-story-11]decide-camera-microphone-and-screen-capture-requests.md new file mode 100644 index 0000000..dd7bab6 --- /dev/null +++ b/requirements/[User-story-11]decide-camera-microphone-and-screen-capture-requests.md @@ -0,0 +1,159 @@ +# Story Decomposition: Decide Camera, Microphone And Screen-Capture Requests + +## INVEST Analysis + +### Abstract Task: "The Application Answers When A Page Asks For The Camera" + +**Analysis Dimensions**: +- **Core Responsibility**: Let an application decide, per request, whether a page it hosts may use the + camera, the microphone or a screen capture, and make the engine's WebRTC support available where the + engine can provide it. + - Today no engine asks the application. WebKitGTK refuses every request when nobody answers it, so on + Linux a page can never get the camera. WebView2 shows its own prompt. WKWebView shows its own prompt + (and a host app that lacks the camera/microphone usage descriptions crashes on first access). +- **Primary Operations**: + - install a permission handler on a component; + - receive each media-capture request (the requesting origin, the page address, and which devices it + wants: camera, microphone, screen) and answer allow, deny, or "do what the engine would do"; + - answer later, from any thread, after the application has asked its user or checked its policy; + - find out whether the loaded native library routes these requests, and (from the page) whether the + engine has WebRTC at all. +- **Key Constraints**: + - All three engines: WebKitGTK (Linux, lightweight component), WebView2 (Windows), WKWebView (macOS 12+). + - The answer may take a person's time, so it must not hold any engine or Swing thread while it waits. + - Without a handler, each engine behaves exactly as it does today. + - Older engines that lack a capability (WebKitGTK before 2.42 has no screen-capture request; macOS + before 12 has no hook) keep working, with the gap documented. +- **Technical Complexity**: High. Three native implementations and an asynchronous answer from Java + back into each engine. +- **Business Complexity**: Low. One handler, three answers. + +### INVEST Evaluation +- ✅ **Independent**: builds on the existing engine wrappers only. +- ✅ **Negotiable**: which facts the request carries; whether WebRTC is switched on by default (it is + switched on, since it only adds capability a page must still be granted devices for). +- ✅ **Valuable**: video chat and voice features in hosted pages, with the application in control. +- ✅ **Estimable**: one permission hook per engine, following the existing per-engine callback pattern. +- ⚠️ **Small**: about 4 days across three engines. It stays one story, like the navigation decision, + because the value is one portable decision point. +- ✅ **Testable**: on Linux, a page can ask for the camera against WebKitGTK's mock devices and report + whether it received a stream or an error. + +**Conclusion**: **Ready as-is — one story.** + +### Split Strategy +Not applicable — no split needed. + +--- + +## [STORY-011-001] Decide camera, microphone and screen-capture requests + +### Background + +The Agentic App Framework hosts small multiplayer web apps ("creations") in this library's component. +Its next feature is video chat between the people in a room: each page asks for the camera and +microphone (`getUserMedia`), optionally shares its screen (`getDisplayMedia`), and connects peer to peer +with WebRTC. + +None of that can work at the desk today: +- on Linux, WebKitGTK refuses every media request nobody answers, so the page always gets + `NotAllowedError`; +- on Windows, WebView2 shows its own generic prompt that the application cannot style, pre-answer or + remember; +- on macOS, WKWebView shows its own prompt, and the application cannot apply its own policy. + +The framework needs one portable decision point: the page asks, the application decides (often after +asking its user in its own UI), and the page gets a stream or a clean refusal. + +### Business Value +- Provide **application developers** with control over which hosted pages may use the camera, + microphone and screen, on every platform. +- Support **video and voice features** in hosted pages on Linux, where they are impossible today. +- Enable **the application's own consent UI** and remembered choices instead of engine prompts. + +### Dependencies and Assumptions +- **Prerequisites**: none beyond the existing component modes and custom schemes (a page served from a + registered custom scheme is a secure context, which media capture requires). +- **Data assumptions**: the machine has the devices; tests on Linux use WebKitGTK's mock capture devices. +- **Integration points**: WebKitGTK, WebView2 and WKWebView permission APIs; the operating system's own + privacy controls still apply on top of the application's answer (macOS camera/microphone consent; + Windows privacy settings). +- **Business constraints**: macOS host apps must declare camera and microphone usage descriptions, or + the OS ends the process when a page first uses the device. This must be documented. + +### Scope In +- A permission handler on the component, told the origin, the page address and the requested kinds + (camera, microphone, screen), answering allow, deny or default — immediately or later, from any thread. +- Linux: requests from the lightweight component reach the handler; WebRTC and media streams are + switched on in the engine; how a page detects whether WebRTC is present is documented. +- Windows: camera and microphone requests reach the handler. +- macOS 12+: camera and microphone requests reach the handler. +- A demo per OS, a Linux smoke test, README documentation of each engine's default and gaps. + +### Scope Out +- Geolocation, notifications, clipboard and other permission kinds (left to the engine). +- Choosing which camera or microphone, or which screen, is shared (the engine's picker, if any). +- Remembering answers between runs (the application can do that in its handler). +- Screen capture on Windows (WebView2 1.0.2592 has no permission kind for it) and on macOS (no public + WKWebView hook); both keep the engine's own behaviour. +- Supplying WebRTC on a Linux distribution whose WebKitGTK was built without it. + +### Acceptance Criteria + +#### AC1: Allowed camera and microphone request gets a stream +**Given** a component on Linux showing `mediatest://app/` with a handler that allows every request +**When** the page asks for `{video: true, audio: true}` +**Then** the page receives a stream with one video and one audio track, and the handler was told the +origin `mediatest://app` and the kinds camera and microphone. + +#### AC2: Denied request is refused cleanly +**Given** the same page with a handler that denies every request +**When** the page asks for the camera +**Then** the page's request fails with `NotAllowedError`, and the handler was told the kind camera only. + +#### AC3: Answering later +**Given** a handler that answers allow two seconds after it is asked, from a background thread +**When** the page asks for the microphone +**Then** the page waits and then receives an audio stream; no window freezes while it waits. + +#### AC4: No handler keeps today's behaviour +**Given** no handler is installed +**When** the page asks for the camera +**Then** Linux refuses it (`NotAllowedError`), Windows shows WebView2's own prompt, macOS shows WebKit's +own prompt — exactly as before this feature. + +#### AC5: Default answer +**Given** a handler that answers "default" +**When** the page asks for the camera +**Then** the outcome is the same as with no handler (AC4). + +#### AC6: A failing handler refuses +**Given** a handler that throws +**When** the page asks for the camera +**Then** the request is refused, and the error reaches the application's uncaught-exception handler. + +#### AC7: An unanswered request does not hang forever +**Given** a handler that never answers +**When** the component is disposed +**Then** the pending request is refused and nothing crashes. + +#### AC8: Only the first answer counts +**Given** a handler that answers allow and then deny +**When** the page asks for the camera +**Then** the page receives a stream; the second answer is ignored. + +#### AC9: Feature detection +**Given** any platform +**When** the application asks whether permission handling is supported +**Then** it is told yes with a native built from this feature, no with an older one; and a page can +detect whether the engine provides WebRTC (on Ubuntu 24.04's WebKitGTK 2.52 build it does not, even with +WebRTC switched on), so it can fall back to an audio/video-free experience or an external browser. + +#### AC10: Screen capture on Linux +**Given** WebKitGTK 2.42 or newer and a handler that allows +**When** the page calls `getDisplayMedia` +**Then** the handler is told the kind screen. + +#### Non-Functional Expectations +- No engine thread or the Swing event thread is blocked while the application decides. +- An answer that arrives after the page or component is gone is ignored safely. diff --git a/run-linux-media-permission-demo.sh b/run-linux-media-permission-demo.sh new file mode 100755 index 0000000..9ef85ea --- /dev/null +++ b/run-linux-media-permission-demo.sh @@ -0,0 +1,181 @@ +#!/bin/bash +# One-shot script: build the Linux native lib, build WebView.jar, compile +# and run the media permission demo (WebViewMediaPermissionDemo, Canvas 35). +# +# Serves a page from demo://app/ whose buttons ask for the camera, the +# microphone and a screen capture. The application's +# WebViewPermissionHandler answers each request (by default after asking you +# in a Swing dialog). See demos/WebViewMediaPermissionDemo/README.md. +# +# Usage: ./run-linux-media-permission-demo.sh # lightweight, the only Linux mode +# MOCK=1 ./run-linux-media-permission-demo.sh # WebKitGTK's mock devices, no camera needed +# +# Override JDK: JAVA_HOME=/path/to/jdk ./run-linux-media-permission-demo.sh +# +# Required packages: +# - g++, pkg-config +# - libgtk-3-dev +# - libwebkit2gtk-4.1-dev (Ubuntu 24.04+) or libwebkit2gtk-4.0-dev (older) +# - libx11-dev +# - libxt-dev <-- pulled in because JDK 8's jawt_md.h includes X11/Intrinsic.h +set -e +set -o pipefail + +REPO_DIR="$(cd "$(dirname "$0")" && pwd)" +cd "$REPO_DIR" + +# --------------------------------------------------------------------------- +# 0. Mode: lightweight only. On Linux we only use the lightweight component +# (see CLAUDE.md); heavyweight is for macOS and Windows. +# --------------------------------------------------------------------------- +MODE=lightweight +echo "Mode: $MODE" + +# --------------------------------------------------------------------------- +# 1. Resolve JAVA_HOME +# --------------------------------------------------------------------------- +if [ -z "$JAVA_HOME" ]; then + if command -v javac >/dev/null 2>&1; then + JAVAC_PATH="$(readlink -f "$(command -v javac)" 2>/dev/null || command -v javac)" + JAVA_HOME="$(dirname "$(dirname "$JAVAC_PATH")")" + fi +fi +if [ -z "$JAVA_HOME" ] || [ ! -d "$JAVA_HOME" ]; then + echo "JAVA_HOME is not set and could not be resolved automatically." >&2 + echo "Install a JDK and export JAVA_HOME, e.g. via mise / asdf:" >&2 + echo " mise install # picks up .tool-versions" >&2 + exit 1 +fi +export JAVA_HOME +echo "Using JAVA_HOME=$JAVA_HOME" +JAVAC="$JAVA_HOME/bin/javac" +JAVA="$JAVA_HOME/bin/java" +JAR="$JAVA_HOME/bin/jar" +for tool in "$JAVAC" "$JAVA" "$JAR"; do + if [ ! -x "$tool" ]; then + echo "Missing tool: $tool" >&2; exit 1 + fi +done + +# --------------------------------------------------------------------------- +# 2. Pick a WebKit pkg-config name +# --------------------------------------------------------------------------- +if pkg-config --exists webkit2gtk-4.1; then + WEBKIT_PKG=webkit2gtk-4.1 +elif pkg-config --exists webkit2gtk-4.0; then + WEBKIT_PKG=webkit2gtk-4.0 +else + echo "Neither webkit2gtk-4.1 nor webkit2gtk-4.0 was found via pkg-config." >&2 + echo "Install one of the following with your package manager:" >&2 + echo " Debian/Ubuntu 24.04+: sudo apt install libgtk-3-dev libwebkit2gtk-4.1-dev libx11-dev libxt-dev" >&2 + echo " Debian/Ubuntu older: sudo apt install libgtk-3-dev libwebkit2gtk-4.0-dev libx11-dev libxt-dev" >&2 + echo " Fedora: sudo dnf install gtk3-devel webkit2gtk4.1-devel libX11-devel libXt-devel" >&2 + exit 1 +fi +if ! pkg-config --exists gtk+-3.0; then + echo "gtk+-3.0 (libgtk-3-dev) is required." >&2 + exit 1 +fi +echo "Using WebKit package: $WEBKIT_PKG" + +if ! printf '#include \nint main(){return 0;}\n' | \ + g++ -E -x c++ - >/dev/null 2>&1; then + echo "" >&2 + echo "ERROR: not found." >&2 + echo "" >&2 + echo "JDK 8 on Linux pulls X11 Intrinsics in via jawt_md.h. Install:" >&2 + echo " Debian/Ubuntu: sudo apt install libxt-dev" >&2 + echo " Fedora: sudo dnf install libXt-devel" >&2 + echo " Arch: sudo pacman -S libxt" >&2 + echo "" >&2 + exit 1 +fi + +# --------------------------------------------------------------------------- +# 3. Build libwebview.so for the current arch +# --------------------------------------------------------------------------- +case "$(uname -m)" in + x86_64|amd64) NATIVE_DIR=linux_64 ;; + aarch64|arm64) NATIVE_DIR=linux_arm64 ;; + armv7l|armv7) NATIVE_DIR=linux_arm ;; + i?86) NATIVE_DIR=linux_32 ;; + *) NATIVE_DIR=linux_64 ;; +esac +echo "Native dir: $NATIVE_DIR (arch: $(uname -m))" +mkdir -p "$REPO_DIR/src/$NATIVE_DIR" +SO="$REPO_DIR/src/$NATIVE_DIR/libwebview.so" + +NEED_SO=0 +if [ ! -f "$SO" ]; then + NEED_SO=1 +else + for src in src_c/webview.c src_c/webview.h src_c/webview_embed.cpp src_c/webkit_loader.cpp src_c/webkit_loader.h src_c/webkit_shim.h; do + if [ "$src" -nt "$SO" ]; then NEED_SO=1; break; fi + done +fi + +if [ "$NEED_SO" = "1" ]; then + echo "Building libwebview.so ..." + g++ -fPIC -std=c++11 -Wall \ + -DWEBVIEW_GTK=1 \ + -I"$JAVA_HOME/include" -I"$JAVA_HOME/include/linux" \ + -I./src_c \ + $(pkg-config --cflags gtk+-3.0 "$WEBKIT_PKG") \ + src_c/webview.c src_c/webview_embed.cpp src_c/webkit_loader.cpp \ + $(pkg-config --libs gtk+-3.0) \ + -lX11 -ldl \ + -shared -o "$SO" + echo "Built $SO" +else + echo "libwebview.so up to date." +fi + +# --------------------------------------------------------------------------- +# 4. Compile WebView Java sources +# --------------------------------------------------------------------------- +BUILD_DIR="$REPO_DIR/build" +WV_CLASSES="$BUILD_DIR/classes-webview" +WV_JAR="$REPO_DIR/dist/WebView.jar" +mkdir -p "$WV_CLASSES" "$REPO_DIR/dist" +echo "Compiling WebView Java sources ..." +find src -name '*.java' > "$BUILD_DIR/wv-sources.txt" +"$JAVAC" -d "$WV_CLASSES" -classpath "lib/*" @"$BUILD_DIR/wv-sources.txt" + +# --------------------------------------------------------------------------- +# 5. Build WebView.jar -- classes + linux_*/libwebview.so at jar root +# --------------------------------------------------------------------------- +echo "Building WebView.jar ..." +STAGE="$BUILD_DIR/stage-webview" +rm -rf "$STAGE" +mkdir -p "$STAGE/$NATIVE_DIR" +cp -R "$WV_CLASSES"/. "$STAGE"/ +cp "$SO" "$STAGE/$NATIVE_DIR/libwebview.so" +( cd "$STAGE" && "$JAR" cf "$WV_JAR" . ) +echo "Built $WV_JAR" + +# --------------------------------------------------------------------------- +# 6. Compile and run the media permission demo +# --------------------------------------------------------------------------- +DEMO_DIR="$REPO_DIR/demos/WebViewMediaPermissionDemo" +DEMO_CLASSES="$BUILD_DIR/classes-media-permission-demo" +mkdir -p "$DEMO_CLASSES" +echo "Compiling demo ..." +"$JAVAC" -d "$DEMO_CLASSES" -classpath "$WV_JAR" \ + "$DEMO_DIR/src/ca/weblite/webview/demos/WebViewMediaPermissionDemo.java" + +echo "Launching WebViewMediaPermissionDemo (mode=$MODE) ..." +# Force the X11 GDK backend; embedding via XReparentWindow needs a real X11 +# GdkDisplay on both sides. +export GDK_BACKEND=x11 +export WEBKIT_DISABLE_DMABUF_RENDERER=1 +export WEBKIT_DISABLE_SANDBOX_THIS_IS_DANGEROUS=1 +# Canvas 35 D7c: MOCK=1 turns on WebKitGTK's mock camera, microphone and +# screen devices (a test-only switch). +if [ "${MOCK:-}" = "1" ]; then + export WEBVIEW_MOCK_CAPTURE_DEVICES=1 +fi + +exec "$JAVA" \ + "-Dca.weblite.webview.mode=$MODE" \ + -cp "$DEMO_CLASSES:$WV_JAR" \ + ca.weblite.webview.demos.WebViewMediaPermissionDemo diff --git a/run-linux-media-smoketest.sh b/run-linux-media-smoketest.sh index d60a85a..17b5d5c 100755 --- a/run-linux-media-smoketest.sh +++ b/run-linux-media-smoketest.sh @@ -1,8 +1,13 @@ #!/bin/bash # One-shot script: build the Linux native lib, build WebView.jar, compile -# and run the media smoke test (Canvas 33). No Ant required. Exits with the -# test's own code: 0 when HTML media plays in the lightweight engine, 1 when -# it does not. Runs headless under Xvfb, e.g.: +# and run the media smoke tests. No Ant required. +# 1. WebViewMediaSmokeTest (Canvas 33): HTML media plays in the lightweight +# engine. +# 2. WebViewMediaPermissionSmokeTest (Canvas 35): camera / microphone / +# screen-capture requests reach a WebViewPermissionHandler, DENY gives +# NotAllowedError and ALLOW gives a stream -- against WebKitGTK's mock +# capture devices (WEBVIEW_MOCK_CAPTURE_DEVICES=1), so no camera needed. +# Exits 0 when both pass, 1 when either fails. Runs headless under Xvfb, e.g.: # xvfb-run -a ./run-linux-media-smoketest.sh # # Usage: ./run-linux-media-smoketest.sh @@ -161,20 +166,36 @@ cp "$SO" "$STAGE/$NATIVE_DIR/libwebview.so" echo "Built $WV_JAR" # --------------------------------------------------------------------------- -# 6. Compile and run the media smoke test +# 6. Compile and run the media smoke tests # --------------------------------------------------------------------------- DEMO_DIR="$REPO_DIR/demos/WebViewMediaSmokeTest" DEMO_CLASSES="$BUILD_DIR/classes-media-smoke" mkdir -p "$DEMO_CLASSES" -echo "Compiling smoke test ..." +echo "Compiling smoke tests ..." "$JAVAC" -d "$DEMO_CLASSES" -classpath "$WV_JAR" \ - "$DEMO_DIR/src/ca/weblite/webview/demos/WebViewMediaSmokeTest.java" + "$DEMO_DIR/src/ca/weblite/webview/demos/WebViewMediaSmokeTest.java" \ + "$DEMO_DIR/src/ca/weblite/webview/demos/WebViewMediaPermissionSmokeTest.java" -echo "Running smoke test ..." # Force the X11 GDK backend (the native pump does too) and disable the # DMA-BUF renderer, which fails silently in virtualized environments. export GDK_BACKEND=x11 export WEBKIT_DISABLE_DMABUF_RENDERER=1 -exec "$JAVA" \ - -cp "$DEMO_CLASSES:$WV_JAR" \ - ca.weblite.webview.demos.WebViewMediaSmokeTest + +echo "Running media playback smoke test (Canvas 33) ..." +set +e +"$JAVA" -cp "$DEMO_CLASSES:$WV_JAR" ca.weblite.webview.demos.WebViewMediaSmokeTest +PLAYBACK=$? + +echo "Running media permission smoke test (Canvas 35) ..." +# Canvas 35 D7c: a test-only switch that turns on WebKitGTK's mock camera, +# microphone and screen devices. +WEBVIEW_MOCK_CAPTURE_DEVICES=1 \ + "$JAVA" -cp "$DEMO_CLASSES:$WV_JAR" ca.weblite.webview.demos.WebViewMediaPermissionSmokeTest +PERMISSION=$? +set -e + +echo "playback smoke test: exit $PLAYBACK; permission smoke test: exit $PERMISSION" +if [ "$PLAYBACK" -ne 0 ] || [ "$PERMISSION" -ne 0 ]; then + exit 1 +fi +exit 0 diff --git a/run-mac-media-permission-demo.sh b/run-mac-media-permission-demo.sh new file mode 100755 index 0000000..ddfba21 --- /dev/null +++ b/run-mac-media-permission-demo.sh @@ -0,0 +1,125 @@ +#!/bin/bash +# One-shot script: build the macOS native lib, build WebView.jar, compile +# and run the media permission demo (WebViewMediaPermissionDemo, Canvas 35). +# +# Serves a page from demo://app/ whose buttons ask for the camera and the +# microphone, in the heavyweight component (the supported macOS mode). The +# application's WebViewPermissionHandler answers each request (by default +# after asking you in a Swing dialog). Needs macOS 12+. +# +# Info.plist: macOS attributes camera / microphone use to the launching +# process. This script runs a plain `java`, so the consent prompt names your +# terminal (or the JDK), which supplies the usage descriptions; a packaged +# host app must declare NSCameraUsageDescription and +# NSMicrophoneUsageDescription itself, or macOS ends it on first access. +# See demos/WebViewMediaPermissionDemo/README.md. +# +# Usage: ./run-mac-media-permission-demo.sh +# Override: JAVA_HOME=/path/to/jdk ./run-mac-media-permission-demo.sh +# +set -e + +REPO_DIR="$(cd "$(dirname "$0")" && pwd)" +cd "$REPO_DIR" + +# --------------------------------------------------------------------------- +# 1. Resolve JAVA_HOME +# --------------------------------------------------------------------------- +if [ -z "$JAVA_HOME" ]; then + if [ -x /usr/libexec/java_home ]; then + JAVA_HOME="$(/usr/libexec/java_home)" + fi +fi +if [ -z "$JAVA_HOME" ] || [ ! -d "$JAVA_HOME" ]; then + echo "JAVA_HOME is not set and could not be resolved automatically." >&2 + echo "Install a JDK and export JAVA_HOME, e.g.:" >&2 + echo " brew install --cask temurin" >&2 + echo " export JAVA_HOME=\"\$(/usr/libexec/java_home)\"" >&2 + exit 1 +fi +export JAVA_HOME +echo "Using JAVA_HOME=$JAVA_HOME" +JAVAC="$JAVA_HOME/bin/javac" +JAVA="$JAVA_HOME/bin/java" +JAR="$JAVA_HOME/bin/jar" +for tool in "$JAVAC" "$JAVA" "$JAR"; do + if [ ! -x "$tool" ]; then + echo "Missing tool: $tool" >&2; exit 1 + fi +done + +# --------------------------------------------------------------------------- +# 2. Build libwebview.dylib if missing or source is newer +# --------------------------------------------------------------------------- +case "$(uname -m)" in + arm64|aarch64) NATIVE_DIR=osx_arm64 ;; + x86_64) NATIVE_DIR=osx_64 ;; + *) NATIVE_DIR=osx_64 ;; +esac +echo "Native dir: $NATIVE_DIR" +mkdir -p "$REPO_DIR/src/$NATIVE_DIR" +DYLIB="$REPO_DIR/src/$NATIVE_DIR/libwebview.dylib" +NEED_DYLIB=0 +if [ ! -f "$DYLIB" ]; then + NEED_DYLIB=1 +else + for src in src_c/webview.c src_c/webview.h src_c/webview_embed.cpp; do + if [ "$src" -nt "$DYLIB" ]; then NEED_DYLIB=1; break; fi + done +fi + +if [ "$NEED_DYLIB" = "1" ]; then + echo "Building libwebview.dylib (arch: $(uname -m)) ..." + c++ \ + -I"$JAVA_HOME/include" -I"$JAVA_HOME/include/darwin" \ + -dynamiclib \ + src_c/webview_embed.cpp \ + -o "$DYLIB" \ + -DWEBVIEW_COCOA=1 \ + -std=c++11 \ + -framework WebKit -framework Cocoa -framework QuartzCore \ + -Wl,-undefined,dynamic_lookup + echo "Built $DYLIB" +else + echo "libwebview.dylib up to date." +fi + +# --------------------------------------------------------------------------- +# 3. Compile WebView Java sources +# --------------------------------------------------------------------------- +BUILD_DIR="$REPO_DIR/build" +WV_CLASSES="$BUILD_DIR/classes-webview" +WV_JAR="$REPO_DIR/dist/WebView.jar" +mkdir -p "$WV_CLASSES" "$REPO_DIR/dist" +echo "Compiling WebView Java sources ..." +find src -name '*.java' > "$BUILD_DIR/wv-sources.txt" +"$JAVAC" -d "$WV_CLASSES" \ + -classpath "lib/*" @"$BUILD_DIR/wv-sources.txt" + +# --------------------------------------------------------------------------- +# 4. Build WebView.jar -- classes + osx_*/libwebview.dylib at the jar root +# --------------------------------------------------------------------------- +echo "Building WebView.jar ..." +STAGE="$BUILD_DIR/stage-webview" +rm -rf "$STAGE" +mkdir -p "$STAGE/$NATIVE_DIR" +cp -R "$WV_CLASSES"/. "$STAGE"/ +cp "$DYLIB" "$STAGE/$NATIVE_DIR/libwebview.dylib" +( cd "$STAGE" && "$JAR" cf "$WV_JAR" . ) +echo "Built $WV_JAR" + +# --------------------------------------------------------------------------- +# 5. Compile and run the media permission demo +# --------------------------------------------------------------------------- +DEMO_DIR="$REPO_DIR/demos/WebViewMediaPermissionDemo" +DEMO_CLASSES="$BUILD_DIR/classes-media-permission-demo" +mkdir -p "$DEMO_CLASSES" +echo "Compiling demo ..." +"$JAVAC" -d "$DEMO_CLASSES" \ + -classpath "$WV_JAR" \ + "$DEMO_DIR/src/ca/weblite/webview/demos/WebViewMediaPermissionDemo.java" + +echo "Launching WebViewMediaPermissionDemo ..." +exec "$JAVA" \ + -cp "$DEMO_CLASSES:$WV_JAR" \ + ca.weblite.webview.demos.WebViewMediaPermissionDemo diff --git a/run-windows-media-permission-demo.bat b/run-windows-media-permission-demo.bat new file mode 100644 index 0000000..bc325d4 --- /dev/null +++ b/run-windows-media-permission-demo.bat @@ -0,0 +1,215 @@ +@echo off +REM ============================================================================ +REM run-windows-media-permission-demo.bat +REM +REM One-shot build & launch of WebViewMediaPermissionDemo on Windows (Canvas 35). +REM Builds the WebView2-backed native DLL, packages WebView.jar, compiles the +REM demo, and runs it. No Ant required. +REM +REM Serves a page from demo://app/ whose buttons ask for the camera and the +REM microphone through WebView2 in the heavyweight component. The +REM application's WebViewPermissionHandler answers each request (by default +REM after asking you in a Swing dialog). Screen sharing keeps WebView2's own +REM picker. See demos\WebViewMediaPermissionDemo\README.md. +REM +REM Requires: +REM - JAVA_HOME set to a JDK 8+ install. +REM - Visual Studio 2019 or newer with "Desktop development with C++". +REM - The WebView2 runtime (ships with Windows 11; install Evergreen on +REM older Windows: https://aka.ms/webview2 ). +REM ============================================================================ +setlocal enabledelayedexpansion + +set "REPO_DIR=%~dp0" +if "%REPO_DIR:~-1%"=="\" set "REPO_DIR=%REPO_DIR:~0,-1%" +echo Repo: %REPO_DIR% + +REM --------------------------------------------------------------------------- +REM 1. Resolve JAVA_HOME +REM --------------------------------------------------------------------------- +if "%JAVA_HOME%"=="" ( + echo. + echo JAVA_HOME is not set. Set it to your JDK install and re-run, e.g.: + echo. + echo set "JAVA_HOME=C:\Program Files\Amazon Corretto\jdk1.8.0_xxx" + echo run-windows-media-permission-demo.bat + echo. + exit /b 1 +) +if not exist "%JAVA_HOME%\bin\javac.exe" ( + echo ERROR: javac.exe not found in "%JAVA_HOME%\bin". + echo Check that JAVA_HOME points at the JDK root, not the JRE. + exit /b 1 +) +echo Using JAVA_HOME=%JAVA_HOME% + +REM --------------------------------------------------------------------------- +REM 2. Locate Visual Studio via vswhere +REM --------------------------------------------------------------------------- +set "vswhere=%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" +if not exist "%vswhere%" set "vswhere=%ProgramFiles%\Microsoft Visual Studio\Installer\vswhere.exe" +if not exist "%vswhere%" ( + echo ERROR: vswhere.exe not found. + echo Install Visual Studio 2019 or newer with the + echo "Desktop development with C++" workload. + exit /b 1 +) +set "vc_dir=" +for /f "usebackq tokens=*" %%i in (`"%vswhere%" -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath`) do ( + set "vc_dir=%%i" +) +if "%vc_dir%"=="" ( + echo ERROR: Could not locate a Visual Studio installation with VC tools x86/x64. + exit /b 1 +) +echo Visual Studio: %vc_dir% + +REM --------------------------------------------------------------------------- +REM 3. Activate the MSVC x64 environment +REM --------------------------------------------------------------------------- +call "%vc_dir%\Common7\Tools\vsdevcmd.bat" -arch=x64 -host_arch=x64 +if errorlevel 1 ( + echo ERROR: vsdevcmd.bat failed. + exit /b 1 +) + +set "SDK_OK=" +for %%I in ("%INCLUDE:;=";"%") do ( + if exist "%%~I\windows.h" set "SDK_OK=1" +) + +if defined SDK_OK goto :sdk_check_done +set "KITS_ROOT=C:\Program Files (x86)\Windows Kits\10" +set "SDK_VER=" +if not exist "%KITS_ROOT%\Include" goto :sdk_check_done +for /f "delims=" %%V in ('dir /b /ad /on "%KITS_ROOT%\Include" 2^>nul') do ( + if exist "%KITS_ROOT%\Include\%%V\um\windows.h" set "SDK_VER=%%V" +) +if "%SDK_VER%"=="" goto :sdk_check_done +echo NOTE: vsdevcmd did not register a Windows SDK, but one is present +echo on disk. Manually adding Windows SDK %SDK_VER% at %KITS_ROOT%. +set "INCLUDE=%INCLUDE%;%KITS_ROOT%\Include\%SDK_VER%\ucrt;%KITS_ROOT%\Include\%SDK_VER%\um;%KITS_ROOT%\Include\%SDK_VER%\shared;%KITS_ROOT%\Include\%SDK_VER%\winrt;%KITS_ROOT%\Include\%SDK_VER%\cppwinrt" +set "LIB=%LIB%;%KITS_ROOT%\Lib\%SDK_VER%\ucrt\x64;%KITS_ROOT%\Lib\%SDK_VER%\um\x64" +set "SDK_OK=1" +:sdk_check_done + +if defined SDK_OK goto :build_dll +echo. +echo ERROR: Windows SDK headers not found on the cl.exe INCLUDE path. +echo vsdevcmd activated successfully but no SDK provides windows.h. +echo. +echo Fix: open the Visual Studio Installer, click Modify on your VS install, +echo and on the "Individual components" tab tick a "Windows 11 SDK" or +echo "Windows 10 SDK" entry, then re-run this script. +echo. +exit /b 1 +:build_dll + +REM --------------------------------------------------------------------------- +REM 4. Build webview.dll (x64) +REM --------------------------------------------------------------------------- +set "NATIVE_DIR=windows_64" +set "DLL_OUT=%REPO_DIR%\src\%NATIVE_DIR%" +if not exist "%DLL_OUT%" mkdir "%DLL_OUT%" +set "BUILD_DIR=%REPO_DIR%\build" +if not exist "%BUILD_DIR%" mkdir "%BUILD_DIR%" +set "WEBVIEW2_VERSION=1.0.2592.51" +set "WEBVIEW2_DIR=%REPO_DIR%\windows\script\Microsoft.Web.WebView2.%WEBVIEW2_VERSION%" +set "WEBVIEW2_NUPKG=%WEBVIEW2_DIR%\Microsoft.Web.WebView2.%WEBVIEW2_VERSION%.nupkg" +set "WEBVIEW2_URL=https://www.nuget.org/api/v2/package/Microsoft.Web.WebView2/%WEBVIEW2_VERSION%" + +set "PWSH=%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" +if not exist "%PWSH%" set "PWSH=powershell.exe" + +if exist "%WEBVIEW2_DIR%\build\native\include\WebView2.h" goto :webview2_ready +if not exist "%WEBVIEW2_DIR%" mkdir "%WEBVIEW2_DIR%" +if exist "%WEBVIEW2_NUPKG%" goto :webview2_extract +echo Downloading Microsoft.Web.WebView2 %WEBVIEW2_VERSION% from NuGet ... +"%PWSH%" -NoProfile -Command "$ProgressPreference='SilentlyContinue'; [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; Invoke-WebRequest -Uri '%WEBVIEW2_URL%' -OutFile '%WEBVIEW2_NUPKG%' -UseBasicParsing" +if not exist "%WEBVIEW2_NUPKG%" ( + echo ERROR: failed to download WebView2 SDK from %WEBVIEW2_URL%. + exit /b 1 +) +:webview2_extract +echo Extracting Microsoft.Web.WebView2.%WEBVIEW2_VERSION%.nupkg ... +"%PWSH%" -NoProfile -Command "Add-Type -AssemblyName System.IO.Compression.FileSystem; [System.IO.Compression.ZipFile]::ExtractToDirectory('%WEBVIEW2_NUPKG%', '%WEBVIEW2_DIR%')" +if not exist "%WEBVIEW2_DIR%\build\native\include\WebView2.h" ( + echo ERROR: failed to extract WebView2 SDK. + exit /b 1 +) +:webview2_ready + +echo Building webview.dll (x64) ... +cl /nologo ^ + /D "WEBVIEW_API=__declspec(dllexport)" ^ + /D "_SILENCE_EXPERIMENTAL_COROUTINE_DEPRECATION_WARNINGS" ^ + /I "%JAVA_HOME%\include" ^ + /I "%JAVA_HOME%\include\win32" ^ + /I "%REPO_DIR%\windows" ^ + /I "%WEBVIEW2_DIR%\build\native\include" ^ + /std:c++17 /EHsc ^ + /Fo"%BUILD_DIR%\\" ^ + "%REPO_DIR%\windows\webview.cc" ^ + "%REPO_DIR%\windows\webview_embed.cc" ^ + /link /DLL ^ + "%WEBVIEW2_DIR%\build\native\x64\WebView2LoaderStatic.lib" ^ + "%JAVA_HOME%\lib\jawt.lib" ^ + advapi32.lib ole32.lib oleaut32.lib shlwapi.lib shell32.lib user32.lib version.lib winhttp.lib ^ + /OUT:"%BUILD_DIR%\webview.dll" +if errorlevel 1 ( + echo ERROR: cl.exe build failed. + exit /b 1 +) +copy /Y "%BUILD_DIR%\webview.dll" "%DLL_OUT%\webview.dll" >nul +echo Built %DLL_OUT%\webview.dll + +REM --------------------------------------------------------------------------- +REM 5. Compile WebView Java sources +REM --------------------------------------------------------------------------- +set "WV_CLASSES=%BUILD_DIR%\classes-webview" +if exist "%WV_CLASSES%" rmdir /s /q "%WV_CLASSES%" +mkdir "%WV_CLASSES%" +echo Compiling WebView Java sources ... +dir /s /b "%REPO_DIR%\src\*.java" > "%BUILD_DIR%\wv-sources.txt" +"%JAVA_HOME%\bin\javac.exe" -encoding UTF-8 -d "%WV_CLASSES%" -classpath "%REPO_DIR%\lib\*" @"%BUILD_DIR%\wv-sources.txt" +if errorlevel 1 ( + echo ERROR: javac failed. + exit /b 1 +) + +REM --------------------------------------------------------------------------- +REM 6. Build WebView.jar +REM --------------------------------------------------------------------------- +set "WV_JAR=%REPO_DIR%\dist\WebView.jar" +if not exist "%REPO_DIR%\dist" mkdir "%REPO_DIR%\dist" +set "STAGE=%BUILD_DIR%\stage-webview" +if exist "%STAGE%" rmdir /s /q "%STAGE%" +mkdir "%STAGE%\%NATIVE_DIR%" +"%SystemRoot%\System32\xcopy.exe" /e /q /y "%WV_CLASSES%\*" "%STAGE%\" >nul +if errorlevel 1 ( + echo ERROR: xcopy of compiled classes into the jar staging dir failed. + exit /b 1 +) +copy /Y "%DLL_OUT%\webview.dll" "%STAGE%\%NATIVE_DIR%\webview.dll" >nul +pushd "%STAGE%" +"%JAVA_HOME%\bin\jar.exe" cf "%WV_JAR%" . +popd +echo Built %WV_JAR% + +REM --------------------------------------------------------------------------- +REM 7. Compile and launch the media permission demo +REM --------------------------------------------------------------------------- +set "DEMO_DIR=%REPO_DIR%\demos\WebViewMediaPermissionDemo" +set "DEMO_CLASSES=%BUILD_DIR%\classes-media-permission-demo" +if not exist "%DEMO_CLASSES%" mkdir "%DEMO_CLASSES%" +echo Compiling demo ... +"%JAVA_HOME%\bin\javac.exe" -encoding UTF-8 -d "%DEMO_CLASSES%" -classpath "%WV_JAR%" "%DEMO_DIR%\src\ca\weblite\webview\demos\WebViewMediaPermissionDemo.java" +if errorlevel 1 ( + echo ERROR: javac failed on demo. + exit /b 1 +) + +echo Launching WebViewMediaPermissionDemo ... +"%JAVA_HOME%\bin\java.exe" -cp "%DEMO_CLASSES%;%WV_JAR%" ca.weblite.webview.demos.WebViewMediaPermissionDemo + +endlocal diff --git a/spdd/analysis/GGQPA-XXX-202610150300-[Analysis]-decide-camera-microphone-and-screen-capture-requests.md b/spdd/analysis/GGQPA-XXX-202610150300-[Analysis]-decide-camera-microphone-and-screen-capture-requests.md new file mode 100644 index 0000000..638f7a5 --- /dev/null +++ b/spdd/analysis/GGQPA-XXX-202610150300-[Analysis]-decide-camera-microphone-and-screen-capture-requests.md @@ -0,0 +1,141 @@ +# Analysis: Decide Camera, Microphone And Screen-Capture Requests (STORY-011-001) + +> Story: `requirements/[User-story-11]decide-camera-microphone-and-screen-capture-requests.md` +> (the original requirement, unmodified, is that file). +> Evidence: a WebKitGTK 2.52.6 spike (Python/GI probes: a secure custom scheme is a secure context; getUserMedia is refused with no `permission-request` handler and succeeds with one; `RTCPeerConnection` stays undefined with `enable-webrtc` on) and the framework's +> video-chat brief (Part 6, "swingwebview"). + +## 1. Business context + +The Agentic App Framework is adding video chat to its hosted "creations". A creation is served from a +registered custom scheme (a secure context), calls `getUserMedia` / `getDisplayMedia`, and talks +WebRTC to its peers. At the desk this fails or is out of the application's hands on every engine: Linux +always refuses, Windows and macOS show engine prompts the application cannot control. The library has +to offer one portable decision point, answerable after the application has asked its user, and switch +WebRTC on where the engine has it. + +## 2. Domain concepts + +| Concept | Meaning here | +|---|---| +| **Media-capture request** | A page asking for devices: camera, microphone, or a screen/window capture. One request may ask for several kinds (WebKit: camera+microphone together; WebView2: one kind per request). | +| **Kind** | `CAMERA`, `MICROPHONE`, `DISPLAY`. | +| **Origin** | The security origin asking (`scheme://host[:port]`). WKWebView reports it; WebView2 reports the requesting frame's URL; WebKitGTK reports neither, so the top page's address stands in. | +| **Decision** | `ALLOW`, `DENY`, or `DEFAULT` — "do what this engine does with nobody listening". | +| **Pending request** | A request the application has not answered yet. The engine holds the page's promise open; the application may answer from any thread, once. | +| **WebRTC availability** | Whether the engine build has `RTCPeerConnection` at all. A property of the engine build, observable only from a page. | + +**Existing concepts reused:** +- **Per-component dispatcher + JNI callback + global ref set at attach** (popups, navigation, downloads: + `NavigationDispatcher`, `WebViewNavigationCallback`, `webview_*_set_navigation_callback`). +- **Asynchronous answer by id from any thread** (custom schemes, Canvas 30–32: `SchemeDispatcher` keeps + a pending map; native keeps the engine's request object by id; `webview_scheme_respond(id, …)` + marshals the answer to the engine thread; first answer wins). +- **Deferral patterns** already used per engine: WebView2 `GetDeferral` + `post_to_worker_thread` + (scheme, dialogs, downloads); WKWebView block `copy` + `dispatch_async` main (dialogs); GTK + `g_object_ref` + `GtkPump::run_async`. +- **Runtime loader** (`webkit_loader.h` X-macros, optional groups gated by `WEBKIT_CHECK_VERSION` and + `WK_HAS`, Canvas 31 D5) for any new WebKitGTK symbol. +- **Origin canonicalisation** (`Origins.canonical`). + +**New concepts:** the permission handler, the request carrier with its one-shot answer, the kind and +decision enums, the JNI-facing permission callback, and a process-wide pending-request table per native +backend. + +**Key business rules:** +- No handler → no change in behaviour, on every engine. +- `DEFAULT` → the same outcome as no handler. +- First answer wins; later answers are ignored. +- A handler that throws refuses; a disposed component refuses its pending requests. +- Nothing blocks an engine thread or the EDT while the application decides. + +## 3. Strategic approach + +### Solution direction +Native permission hook (per engine) → JNI callback on the engine thread → Java dispatcher. The +dispatcher answers `DEFAULT` synchronously when no handler is installed (native then leaves the engine +untouched — the exact pre-feature path); otherwise it records the request as pending, returns "pending" +to native at once, and runs the handler on the EDT with `invokeLater` (the library's convention for +asynchronous, user-facing callbacks: `evalAsync` continuations, download notifications). The handler +answers through the request object, now or later, from any thread; the answer goes to a static native +`respond(id, decision)` which marshals to the engine thread and completes the engine's request. + +### Key design decisions +- **Handler shape: a request object with `allow()/deny()/useDefault()`**, like + `WebViewSchemeResponder`, rather than a returned future → one answering idiom across the library; + works equally for "answer now" and "answer after a dialog". A `CompletionStage` adapter is easy for + callers to write and isn't needed in the API. +- **Handler runs on the EDT (invokeLater)** → applications typically show a Swing consent dialog. Unlike + the synchronous dialog/popup/navigation hooks, nothing waits on it, so there's no deadlock hazard. +- **Synchronous fast path for "no handler"** → keeps today's behaviour byte-for-byte; in particular + WKWebView's and WebView2's own prompts and WebKitGTK's own denial. +- **`DEFAULT` after pending** → GTK: deny (what WebKitGTK's default handler does once the signal is + claimed); WebView2: leave `State` at `DEFAULT` and complete the deferral (WebView2 then applies its own + default — its prompt); WKWebView: `WKPermissionDecisionPrompt`. +- **Linux settings**: switch on `enable-media-stream` and `enable-webrtc` on every GTK view, by GObject + property lookup (`g_object_class_find_property`) so older WebKitGTK without a property simply skips + it. WebRTC can't be detected from settings (the spike shows `enable-webrtc` reads back TRUE on a build + without WebRTC), so detection is documented as a page check (`typeof RTCPeerConnection`). +- **Mock capture devices for tests**: `enable-mock-capture-devices`, switched on only when the + environment variable `WEBVIEW_MOCK_CAPTURE_DEVICES=1` is set — a test hook, not API. +- **Screen capture**: Linux through `webkit_user_media_permission_is_for_display_device` (2.42+, + optional). Windows (SDK 1.0.2592.51 has no permission kind for it) and macOS (no public hook) keep the + engine's own behaviour; documented. +- **Release version**: no Canvas or norm ties a feature to a version bump (`pom.xml` stays + `1.0-SNAPSHOT`); none is made. + +### Alternatives considered +- **Synchronous boolean on the engine thread** (the navigation pattern) — rejected: a consent prompt + needs a person; blocking the GTK/AppKit/WebView2 thread for that freezes every view, and on macOS + deadlocks with the EDT. +- **Handler returns `CompletableFuture`** — workable, but a null/never-completed future is + easier to get wrong than a responder object, and it diverges from the scheme responder. +- **A native "is WebRTC available" probe** — no cheap, reliable native signal exists (WebKitGTK exports + nothing WebRTC-specific; the setting reads back TRUE regardless). Rejected for a documented page check. + +## 4. Risk & gap analysis + +### Requirement ambiguities +- **Origin on Linux** is the top page's origin, not the requesting frame's — WebKitGTK's + `WebKitUserMediaPermissionRequest` exposes no origin. Documented. +- **Display + audio**: `getDisplayMedia({audio:true})` asks for system audio, not the microphone; the + request is reported as `DISPLAY` only. + +### Edge cases +- The page navigates away while a request is pending → the engine cancels its own request; a late answer + must be harmless (GTK: answering a stale request is a no-op in WebKit; WebView2/WKWebView: the + deferral/block is still valid to complete). +- Component disposed while pending → Java refuses every pending request; native drains its table for + that engine before the view is destroyed. +- An adopted popup → served by the adopting engine's callback (GTK adoption reuses the engine creation + function; Windows adoption must register the handler on the child too). +- A handler answering twice, or answering `null` → first wins; `null` is treated as `DENY`. + +### Technical risks +- **Lightweight engine on Linux** renders into a hidden `GTK_WINDOW_POPUP` (Canvas 33). The spike saw + `getUserMedia` hang in a `GtkOffscreenWindow`; the popup window counts as "in a window", which should + let capture start. Must be verified by the smoke test. +- **WebKitGTK type checks**: `WEBKIT_IS_USER_MEDIA_PERMISSION_REQUEST` references + `webkit_user_media_permission_request_get_type`, which must come through the loader (optional), never + as a link-time reference. +- **Windows and macOS are untestable here**: written against SDK headers and Apple docs; only compiled + by CI. WebView2's `put_State(DEFAULT)` after a deferral is documented to fall back to the default + behaviour (a prompt) — needs on-device confirmation. +- **macOS Info.plist**: a host app without `NSCameraUsageDescription` / `NSMicrophoneUsageDescription` + is killed by TCC on first access. Documentation only; the library cannot prevent it. +- **CI Linux headers are WebKitGTK 2.44 (Debian 11 container)**: display capture (2.42) compiles in; + older headers compile it out through the version gate. + +### Acceptance-criteria coverage +| AC# | Description | Addressable? | Notes | +|---|---|---|---| +| 1 | Allowed → stream, kinds + origin reported | Yes | Linux smoke test with mock devices. | +| 2 | Denied → `NotAllowedError` | Yes | Smoke test. | +| 3 | Answer later from another thread | Yes | Smoke test answers after a delay from a timer thread. | +| 4 | No handler → engine default | Yes | Linux verified (denial); Windows/macOS by construction, untested. | +| 5 | `DEFAULT` = no handler | Yes | Linux verified. | +| 6 | Throwing handler refuses | Yes | Unit test of the dispatcher. | +| 7 | Dispose refuses pending | Yes | Unit test + native drain. | +| 8 | First answer wins | Yes | Unit test. | +| 9 | Feature detection | Yes | `isPermissionHandlerSupported()`; WebRTC by page check. | +| 10 | Display capture on Linux | Partial | Routed when WebKitGTK ≥ 2.42 raises it; whether Xvfb + mock devices raise a display request is environment-dependent, checked best-effort by the smoke test. | diff --git a/spdd/prompt/35-20261015-0310-[Feat]-Decide-Camera-Microphone-And-Screen-Capture-Requests.md b/spdd/prompt/35-20261015-0310-[Feat]-Decide-Camera-Microphone-And-Screen-Capture-Requests.md new file mode 100644 index 0000000..371c07a --- /dev/null +++ b/spdd/prompt/35-20261015-0310-[Feat]-Decide-Camera-Microphone-And-Screen-Capture-Requests.md @@ -0,0 +1,372 @@ +--- +generated_at: 2026-10-15T03:10:00 +story: STORY-011-001 +epic: requirements/[User-story-11]decide-camera-microphone-and-screen-capture-requests.md +builds_on: + - spdd/prompt/33-20260927-1840-[Feat]-Media-Plays-In-The-Lightweight-Engine.md + - spdd/prompt/34-20260930-1720-[Feat]-Decide-Where-A-Page-May-Navigate.md + - spdd/prompt/30-20260926-1445-[Feat]-Custom-Url-Schemes-And-Macos-Coverage.md +--- + +# REASONS Canvas: Decide Camera, Microphone And Screen-Capture Requests — Java API, Linux, macOS and Windows (011-001) + +> Source story: `requirements/[User-story-11]decide-camera-microphone-and-screen-capture-requests.md` → +> **[STORY-011-001]**. +> Analysis: `spdd/analysis/GGQPA-XXX-202610150300-[Analysis]-decide-camera-microphone-and-screen-capture-requests.md`. +> +> The wiring follows the navigation decision (Canvas 34: handler, dispatcher, JNI callback, one global +> ref per engine installed at attach); the asynchronous answer follows custom schemes (Canvas 30–32: a +> pending table keyed by id, answered from any thread, marshalled to the engine thread, first answer +> wins). All three engines are in one Canvas because the value is one portable decision point. +> **Amends Canvas 33's** `run-linux-media-smoketest.sh`, which now also runs this Canvas's smoke test. + +## REASONS-Implements + +**Java (`src/ca/weblite/webview/`)** +- **New:** + - `WebViewPermissionHandler`: the application's decision (D1); + - `WebViewPermissionRequest`: what the handler is told, and how it answers (D2); + - `MediaCaptureKind`: `CAMERA`, `MICROPHONE`, `DISPLAY`, with a bit-mask wire contract (D2); + - `PermissionDecision`: `DEFAULT`, `ALLOW`, `DENY`, with an ordinal wire contract (D2); + - `WebViewPermissionCallback`: the JNI-facing interface (D5); + - `PermissionDispatcher`: per-component; fast path, pending table, EDT delivery, disposal (D3, D4). +- **Edited:** + - `WebViewNative`: `webview_permission_available`, `webview_permission_respond`, + `webview_embed_set_permission_callback`, `webview_offscreen_set_permission_callback` (D5); + - `EmbeddedWebView` and `OffscreenWebView`: `setPermissionCallback` (D5); + - `swing/WebViewComponent`: `setPermissionHandler`, `getPermissionHandler`, + `isPermissionHandlerSupported()` (D1, D6); + - `swing/WebViewHeavyweightComponent` and `swing/WebViewLightweightComponent`: install the callback at + attach, before the first navigation; dispose the dispatcher (D5). + +**Native** +- **Edited** `src_c/webkit_loader.h` (added to `WK_WEBKIT_OPT_SYMS`, which `webkit_loader.cpp` already + resolves optionally) and `src_c/webkit_shim.h`: optional media-permission symbol groups (D7a). +- **Edited** `src_c/webview_embed.cpp`: the GTK `permission-request` handler and media settings on + `Engine` and `OffEngine` (D7); the Cocoa `requestMediaCapturePermissionForOrigin…` selector (D8); the + shared JNI bridges (D5). +- **Edited** `windows/webview_embed.cc`: `PermissionRequested` on the engine and an adopted child, and the + JNI bridges (D9). + +**Tests (JUnit 4, headless)** +- **New** `test/ca/weblite/webview/PermissionDispatcherTest.java`. + +**Demo, smoke test and docs** +- **New** `demos/WebViewMediaPermissionDemo/` (source + README), `run-linux-media-permission-demo.sh`, + `run-mac-media-permission-demo.sh`, `run-windows-media-permission-demo.bat`. +- **New** `demos/WebViewMediaSmokeTest/src/ca/weblite/webview/demos/WebViewMediaPermissionSmokeTest.java`. +- **Edited** `run-linux-media-smoketest.sh` and `demos/WebViewMediaSmokeTest/README.md` (D11). +- **Edited** `README.md` (a "Camera, microphone and screen capture" section and a demo entry) and + `docs/index.html` (a demo card). + +## Decisions (resolved here) + +- **D1 · `WebViewPermissionHandler`.** + - One method, `void permissionRequested(WebViewPermissionRequest request)`. The handler answers through + the request, now or later, from any thread. + - **No handler is the default.** `getPermissionHandler()` returns `null` until one is set, and + `setPermissionHandler(null)` removes it. With no handler, every request goes to the engine exactly + as before this Canvas (D3 fast path). Unlike the dialog and download handlers there is no library + UI to fall back to, so `null` means "the engine decides", not "refuse". + - **Threading:** the handler runs on the **Swing EDT**, delivered with `SwingUtilities.invokeLater`. + Nothing waits for it to return, so it may show a modal dialog, and it may also return at once and + answer later from any thread. It must not block the EDT for long. + - **Asked about:** camera, microphone and screen-capture requests only. Geolocation, notifications and + every other permission stay with the engine. + +- **D2 · `WebViewPermissionRequest`** (final; its answer state is thread-safe): + - `source()` the component; `origin()` the requesting origin `scheme://host[:port]` (canonicalised by + `Origins.canonical`; derived from `url()` when the engine reports no origin; empty when there is + none, e.g. `about:blank`); `url()` the address the engine reports (never null; may be empty); + `kinds()` an unmodifiable `Set` (never empty for a dispatched request); + `has(MediaCaptureKind)`. + - Answers: `allow()`, `deny()`, `useDefault()` and `answer(PermissionDecision)` (null means `DENY`). + Each returns `true` when it was the first answer and so took effect, `false` otherwise. + `isAnswered()`. + - **`MediaCaptureKind`**: `CAMERA` (bit 1), `MICROPHONE` (bit 2), `DISPLAY` (bit 4); `bit()`, + `static Set fromBits(int)` (unknown bits ignored). + - **`PermissionDecision`**: `DEFAULT` (0), `ALLOW` (1), `DENY` (2). The ordinals are the wire contract; + never reorder. The JNI "pending" answer is `-1` (`PermissionDispatcher.PENDING`), not a decision. + +- **D3 · `PermissionDispatcher.dispatch(long id, String origin, String url, int kindBits)`** returns an + `int`, inline on the calling (native UI) thread, and never blocks: + - disposed → `DENY` (2); + - no handler → `DEFAULT` (0) without building a request — the **fast path**, which tells native to + leave the request to the engine untouched; + - no known kind in `kindBits` → `DEFAULT` (0); + - otherwise: build the request, put it in the pending table under `id`, schedule + `handler.permissionRequested(request)` with `invokeLater`, and return `PENDING` (-1). A handler that + throws has the throwable forwarded to the default uncaught-exception handler and the request + denied (a no-op if it had already answered). + - **The first answer wins** (`AtomicBoolean` CAS); it removes the request from the table and sends + `(id, ordinal)` to the **sink** — `WebViewNative.webview_permission_respond` by default, a recorder in + tests. A sink that throws is reported, never rethrown. + +- **D4 · Disposal.** `disposeAll()` marks the dispatcher disposed and **denies** every pending request + (AC7). Native also drains its own table for a destroyed engine (D7–D9), so a request whose answer never + arrives can't leak or hang the page past its component. + +- **D5 · Wiring.** + - **`WebViewPermissionCallback.onPermissionRequested(long id, String origin, String url, int kinds)`** + returns `int` (`-1` pending, else a `PermissionDecision` ordinal). + - **`WebViewNative`:** `static native boolean webview_permission_available()`; + `static native void webview_permission_respond(long id, int decision)`; + `webview_embed_set_permission_callback(long, WebViewPermissionCallback)`; + `webview_offscreen_set_permission_callback(long, WebViewPermissionCallback)`. + - **`EmbeddedWebView` and `OffscreenWebView`** gain `setPermissionCallback`, anchoring the callback in + `heap` and absorbing `UnsatisfiedLinkError` (an older native keeps working without the feature). + - **Both components install the callback at attach, before the first navigation,** beside the + navigation callback; adopted popups get it the same way, and `dispose` calls + `permissionDispatcher.disposeAll()`. + - **Native contract, all engines:** + 1. Allocate an id from one process-wide counter and record the engine's request (retained) in a + process-wide pending table, tagged with its engine, **before** calling Java. + 2. Call Java on the engine thread. A pending exception is cleared and treated as `DENY`. + 3. On an immediate answer, remove the entry and apply the answer at once. + 4. On `PENDING`, keep it. `webview_permission_respond` (any thread) posts to the engine thread, which + looks the id up **at run time**, removes it and applies the answer. Because the post can only run + after the handler returned, the entry is always complete by then. An unknown id is ignored (first + answer wins; already drained). + 5. On engine destroy, the engine's entries are removed and **denied** on the engine thread before the + view is torn down. + +- **D6 · `WebViewComponent.isPermissionHandlerSupported()`** answers + `WebViewNative.webview_permission_available()`, false on any `Throwable` (an older native). A native + built with this Canvas answers true on all three platforms. + +- **D7 · Linux (WebKitGTK), on `Engine` and `OffEngine`.** + - **D7a · Loader.** A new optional group, `WK_OPT_MEDIA` (headers ≥ 2.8): + `webkit_permission_request_allow`, `webkit_permission_request_deny`, + `webkit_user_media_permission_request_get_type`, `webkit_user_media_permission_is_for_audio_device`, + `webkit_user_media_permission_is_for_video_device`; and `WK_OPT_2_42` (headers ≥ 2.42): + `webkit_user_media_permission_is_for_display_device`. Each gets a shim redirect inside the same + `#if`. `webview_permission_available()` stays true; when the runtime lacks the core four (`allow`, + `deny`, `get_type`, `is_for_audio_device`/`_video_device`), the handler returns FALSE (engine + default) without asking Java. + - **D7b · Signal.** One static function, + `gtk_handle_permission_request(JavaVM*, jobject cb, void *engine, WebKitWebView*, WebKitPermissionRequest*)`, + reached through per-engine wrappers connected to `"permission-request"` at view creation (so adopted + popups, which go through the same creation function, are covered). It: + - returns FALSE unless the request is a user-media request: `G_TYPE_CHECK_INSTANCE_TYPE(request, + webkit_user_media_permission_request_get_type())` through the loader; + - computes kinds: `is_for_display_device` (when resolved) → `DISPLAY` only; otherwise + `is_for_video_device` → `CAMERA`, `is_for_audio_device` → `MICROPHONE`; none → FALSE; + - with no callback → FALSE; + - url = `webkit_web_view_get_uri(view)`, origin = empty (WebKitGTK reports no origin; Java derives it + from the page address); + - follows D5: `g_object_ref` for the table; immediate `DEFAULT` → unref, return FALSE (WebKit's own + handler runs, which denies); immediate `ALLOW`/`DENY` → apply, unref, return TRUE; pending → + return TRUE. + - Applying `DEFAULT` after pending **denies** — WebKitGTK's own default once the signal is claimed. + - **D7c · Settings.** At view creation, on both engines, `gtk_apply_media_settings(WebKitSettings*)` + sets `enable-media-stream` and `enable-webrtc` to TRUE, and `enable-mock-capture-devices` to TRUE + only when the environment variable `WEBVIEW_MOCK_CAPTURE_DEVICES` is `1` (a test hook, not API). + Each is set through `g_object_set` only after `g_object_class_find_property` finds it, so older + WebKitGTK skips what it lacks with no warning and no loader symbol. + - **D7d · WebRTC availability** is not probed natively: `enable-webrtc` reads back TRUE on a build + without WebRTC (Ubuntu 24.04's 2.52.6). A page detects it with `typeof RTCPeerConnection === + 'function'`; documented in the README. + - **D7e · Respond / drain** run on the GTK pump thread (`GtkPump::run_async`). Destroy disconnects the + signal (`g_signal_handlers_disconnect_by_func`) and drains inside the `run_sync` that destroys the + window, before `gtk_widget_destroy`. + - **D7f · Linux is lightweight only.** Verification is in the lightweight component (`OffEngine`, + Canvas 33's hidden popup toplevel). The `Engine` path carries the same hook only so the two GTK + engines stay alike; it is never run, tested or verified on Linux (see `CLAUDE.md`). + +- **D8 · macOS (WKWebView, macOS 12+).** + - The delegate class adds `webView:requestMediaCapturePermissionForOrigin:initiatedByFrame:type:decisionHandler:` + (type `v@:@@@q@`), registered only when `__builtin_available(macOS 12.0, *)`; on older macOS the + selector is never added and WebKit behaves as before. + - Kinds from `WKMediaCaptureType`: 0 → `CAMERA`; 1 → `MICROPHONE`; 2 → both; anything else → answer + Prompt at once. + - Origin from the `WKSecurityOrigin`: `protocol://host` plus `:port` when `port` is non-zero; url from + `frame.request.URL.absoluteString`. + - With no engine, no callback, or a destroyed engine → `decisionHandler(WKPermissionDecisionPrompt)` — + exactly what WebKit does when the delegate doesn't implement the selector. + - Otherwise D5 with the handler block `copy`'d into the table; Java is called on AppKit main (the + dispatcher never blocks). Decisions: `ALLOW` → Grant (1), `DENY` → Deny (2), `DEFAULT` → Prompt (0). + Respond and drain run with `dispatch_async` on the main queue; the block is released after it is + called. + - **Screen capture** has no public WKWebView hook; `getDisplayMedia` keeps WebKit's own behaviour. + - **Info.plist:** a host app must declare `NSCameraUsageDescription` and + `NSMicrophoneUsageDescription`; without them macOS terminates the process when the page first uses + the device. The OS's own consent prompt still appears once per app, on top of the handler's answer. + Documented in the README and the demo's README; the mac run script launches through a plain `java`, + whose own bundle governs (documented). + +- **D9 · Windows (WebView2, SDK 1.0.2592.51).** + - `PermissionRequestedHandler` (`ICoreWebView2PermissionRequestedEventHandler`, `Engine*`) registered + with `add_PermissionRequested` beside `win_register_navigation`, for a fresh engine and an adopted + child; its token is stored on the engine and removed on destroy. + - Kinds: `COREWEBVIEW2_PERMISSION_KIND_CAMERA` → `CAMERA`, `_MICROPHONE` → `MICROPHONE`; any other kind + returns `S_OK` untouched. The SDK has no screen-capture kind; `getDisplayMedia` keeps WebView2's own + picker. + - url = `get_Uri` (the requesting frame's address); origin = empty (derived in Java). + - With no callback → `S_OK` untouched (WebView2's own prompt). + - Otherwise D5: the entry holds `args` (AddRef'd), the engine and the worker thread id. Immediate + answer → `put_State` (`ALLOW` → `COREWEBVIEW2_PERMISSION_STATE_ALLOW`, `DENY` → `_DENY`, `DEFAULT` → + leave it), done. Pending → `GetDeferral` stored in the entry; respond posts to the worker thread + (`post_to_worker_thread`), which puts the state, `Complete()`s and releases. `DEFAULT` after + pending leaves the state at `COREWEBVIEW2_PERMISSION_STATE_DEFAULT`, which WebView2 resolves with + its own behaviour. A failed `GetDeferral` denies at once. + - Drain: `destroy_engine` posts a drain to the worker thread **before** `WM_EMBED_QUIT` (or runs it + through `dispatch_to_thread` for a shared-thread adopted engine), so it runs first. + +- **D10 · Demo.** `WebViewMediaPermissionDemo` serves `demo://app/` from a custom scheme (a secure + context), with buttons for camera, microphone, both, and screen. Its handler shows a Swing + `JOptionPane` (Allow / Deny / Default) for each request — answered asynchronously after the dialog — + and logs origin, url and kinds. The page shows the stream in a `