fix(portal): recover capture sooner after GNOME display changes (#27)

README.en.md

@@ -74,7 +74,7 @@ real hardware, please [open an issue](../../issues) with your distribution, desk
 Wayland limits: single screen, fixed resolution. The native Hyprland backend now pipelines two shm screencopy buffers and passes the viewer's requested fps to both capture pacing and the encoder. Actual fps remains limited by the compositor, rendering and readback (the old serial path measured about 16–22 fps at 2560x1600; the pipeline is VM-measured only, not a hardware guarantee). The portal backend does not support dynamic resolution; see below for GNOME wake and lock recovery; the clipboard syncs while the viewer's remote window has focus. On the first connection the desktop asks for "Remote Desktop / Screen Sharing"
 permission; the permission is remembered once granted.
 
-Scaled GNOME desktops using native Portal input use EIS on the same authorized session when `ConnectToEIS` and a screen `mapping_id` are available. Each axis is mapped using the matched region and actual frame dimensions, with no fixed resolution or scale factor. This avoids scaled-coordinate rejection by `NotifyPointerMotionAbsolute`. Portal builds require libei >= 1.1. Missing EIS support retains the legacy input path and its existing scaling limitations; other initialization failures are reported and trigger session recovery. After a successful EIS connection, all input uses EIS. Capture alone and uinput do not initialize EIS; missing optional logical dimensions do not block capture. Keyboard input uses the existing Windows VK to evdev mapping. Recovery keeps the encoder frame dimensions fixed but accepts logical-size changes after scaling. A previously streaming Portal capture that stays paused without a negotiated format for five seconds enters session recovery; a static desktop does not trigger this timeout. Tested while connected on Ubuntu 26.04, GNOME Wayland, one 3840x2160 display: 200% -> 166.67% -> 200%, with continuously updating video and full-screen pointer control after recovery. Switching briefly freezes and goes black before recovering; five seconds is the detection threshold, not the total recovery time. Lock recovery still needs hardware acceptance testing; multi-monitor support, live resolution changes, and rotation/reflection transforms are outside this fix's scope. Capture settings remain at protocol version 4: advertised resolutions use pixel coordinates and DPR 1; Portal logical dimensions are used only for local input mapping.
+Scaled GNOME desktops using native Portal input use EIS on the same authorized session when `ConnectToEIS` and a screen `mapping_id` are available. Each axis is mapped using the matched region and actual frame dimensions, with no fixed resolution or scale factor. This avoids scaled-coordinate rejection by `NotifyPointerMotionAbsolute`. Portal builds require libei >= 1.1. Missing EIS support retains the legacy input path and its existing scaling limitations; other initialization failures are reported and trigger session recovery. After a successful EIS connection, all input uses EIS. Capture alone and uinput do not initialize EIS; missing optional logical dimensions do not block capture. Keyboard input uses the existing Windows VK to evdev mapping. Recovery keeps the encoder frame dimensions fixed but accepts logical-size changes after scaling. On GNOME, a display-change notification enables earlier recovery when previously streaming Portal capture remains paused without a valid format for a 200 ms grace after both conditions are present. Healthy renegotiation cancels the early check; repeated notifications do not postpone the same interruption. The five-second fallback remains without a related notification. Static video and valid-format pauses do not trigger recovery. Tested while connected on Ubuntu 26.04, GNOME Wayland, one 3840x2160 display: 200% -> 166.67% -> 200%, with continuously updating video and full-screen pointer control after recovery. Switching briefly freezes and goes black before recovering; five seconds is the fallback threshold, not total recovery time. See [#35](../../pull/35) for timing measurements and validation limits. Lock recovery still needs hardware acceptance testing; multi-monitor support, live resolution changes, and rotation/reflection transforms are outside this fix's scope. Capture settings remain at protocol version 4: advertised resolutions use pixel coordinates and DPR 1; Portal logical dimensions are used only for local input mapping.
 
 GNOME Portal CONTROL no longer waits for a consent dialog or the first video frame. It announces the selected desktop before reporting secure-desktop state and receiving the official viewer's “auto unlock host” request. The official Android viewer requires this desktop selection before entering its auto-unlock flow, so the notification must precede capture readiness; actual screen dimensions, codec information and capture permissions still follow their real readiness state. Enable auto unlock in the viewer and use the current Linux user's login password. The host verifies it with PAM, unlocks the existing GNOME session, requests display wake, and restores capture, input and clipboard through the shared Portal session. Initial consent or revoked permissions still require local approval. Reconnect if physical or logical dimensions change.
 

README.md

@@ -66,7 +66,7 @@ GNOME Wayland 的 Portal/EIS 路径已在 Ubuntu 26.04、GNOME 50、单屏 4K/20
 
 Wayland 下的限制：只支持单屏、固定分辨率；Hyprland 原生后端默认使用双缓冲流水线 shm screencopy，控制端帧率设置会同步到采集节拍和编码器，但实际帧率仍受合成器、渲染和读回速度限制（旧串行路径在 2560x1600 曾测得约 16～22 帧；流水线只在虚拟机里测量过，不代表实机保证）；portal 后端暂不支持动态分辨率；GNOME 的屏幕唤醒与锁屏恢复见下；剪贴板只在控制端的远控窗口处于前台时同步。
 
-GNOME 缩放桌面在使用 Portal 原生键鼠时，若支持 `ConnectToEIS` 和屏幕 `mapping_id`，通过同一授权会话的 EIS 接口注入输入，按匹配的区域与实际帧尺寸分别映射横纵坐标，没有固定分辨率或缩放倍率，可避免 `NotifyPointerMotionAbsolute` 在缩放时拒绝坐标。Portal 构建需要 libei ≥ 1.1。缺少 EIS 支持时保留旧输入路径及其既有缩放限制；其他初始化错误会明确报告并触发会话恢复。EIS 连接成功后全部输入统一使用 EIS。只采集画面或使用 uinput 时不会初始化 EIS；缺少可选的逻辑尺寸也不阻断采集。键盘沿用项目现有的 Windows VK → evdev 映射。恢复时保持编码器的采集帧尺寸不变，允许缩放后的逻辑尺寸变化。已开始推流的 Portal 采集若暂停且无有效协商格式持续五秒，将进入会话恢复；静态桌面不会触发该超时。Ubuntu 26.04、GNOME Wayland 单屏 3840×2160 已实测保持连接切换 200%→166.67%→200%，恢复后画面持续更新且鼠标可控制全屏；切换期间会短暂停顿、黑屏后恢复，五秒是故障判定阈值，并非总恢复时间。此路径仍需实机验证锁屏恢复；多屏、连接中改变分辨率及旋转/镜像变换不属于本次修复范围。 捕获设置协议仍为版本 4：对客户端上报的分辨率统一使用像素坐标、DPR 为 1；Portal 逻辑尺寸仅用于本地输入映射。
+GNOME 缩放桌面在使用 Portal 原生键鼠时，若支持 `ConnectToEIS` 和屏幕 `mapping_id`，通过同一授权会话的 EIS 接口注入输入，按匹配的区域与实际帧尺寸分别映射横纵坐标，没有固定分辨率或缩放倍率，可避免 `NotifyPointerMotionAbsolute` 在缩放时拒绝坐标。Portal 构建需要 libei ≥ 1.1。缺少 EIS 支持时保留旧输入路径及其既有缩放限制；其他初始化错误会明确报告并触发会话恢复。EIS 连接成功后全部输入统一使用 EIS。只采集画面或使用 uinput 时不会初始化 EIS；缺少可选的逻辑尺寸也不阻断采集。键盘沿用项目现有的 Windows VK → evdev 映射。恢复时保持编码器的采集帧尺寸不变，允许缩放后的逻辑尺寸变化。GNOME 显示配置变化后，已推流的 Portal 采集若持续「暂停且无有效格式」，会在关联通知后的 200 毫秒宽限结束后提前进入现有恢复流程；正常重协商恢复会取消提前判定，连续通知不会推迟同次异常的判定。未收到相关通知时保留五秒兜底；静态画面或有效格式下的暂停不会触发恢复。Ubuntu 26.04、GNOME Wayland 单屏 3840×2160 已实测保持连接切换 200%→166.67%→200%，恢复后画面持续更新且鼠标可控制全屏；切换期间会短暂停顿、黑屏后恢复，五秒是兜底判定阈值，并非总恢复时间。提前恢复的测量结果和验证范围见 [#35](../../pull/35)。此路径仍需实机验证锁屏恢复；多屏、连接中改变分辨率及旋转/镜像变换不属于本次修复范围。 捕获设置协议仍为版本 4：对客户端上报的分辨率统一使用像素坐标、DPR 为 1；Portal 逻辑尺寸仅用于本地输入映射。
 
 GNOME Portal 的控制通道不再等待授权窗口或第一帧画面：先通知客户端当前选择桌面，再上报锁屏状态、接收官方客户端的「自动解锁被控端」请求。官方 Android 客户端需要桌面目标通知才会进入自动解锁流程，因此该通知不能等待采集就绪；真实屏幕尺寸、编码信息及采集权限仍按实际就绪状态上报。请在客户端启用自动解锁，使用当前 Linux 用户的登录密码；主机通过 PAM 验证后解锁当前 GNOME 会话、请求点亮屏幕，并恢复同一 Portal 会话中的画面、键鼠和剪贴板。首次授权或权限被撤销时仍需本机确认。物理或逻辑分辨率变化后需要重连。
 

c_engine/portal_engine.c

@@ -24,6 +24,9 @@
 #define REQUEST "org.freedesktop.portal.Request"
 #define SESSION "org.freedesktop.portal.Session"
 #define PORTAL_RENEGOTIATION_TIMEOUT (5 * G_TIME_SPAN_SECOND)
+#define PORTAL_DISPLAY_GRACE (200 * G_TIME_SPAN_MILLISECOND)
+#define MUTTER_DISPLAY "org.gnome.Mutter.DisplayConfig"
+#define MUTTER_DISPLAY_PATH "/org/gnome/Mutter/DisplayConfig"
 
 struct uu_portal {
     GDBusConnection *bus;
@@ -33,7 +36,7 @@ struct uu_portal {
     int dbus_started;
     char *session;
     char *restore_token;
-    guint closed_subscription, clipboard_owner_subscription, clipboard_transfer_subscription;
+    guint closed_subscription, clipboard_owner_subscription, clipboard_transfer_subscription, display_subscription;
     int clipboard_enabled, clipboard_changed;
     char *clipboard_text, *clipboard_mime;
     size_t clipboard_length;
@@ -51,7 +54,7 @@ struct uu_portal {
     int error;
     enum pw_stream_state state;
     int format_valid, stream_started;
-    gint64 interrupted_at;
+    gint64 interrupted_at, display_changed_at;
     uint32_t node, width, height;
     int logical_width, logical_height;
     int absolute_stream_pixels;
@@ -176,14 +179,31 @@ static int has_interface(struct uu_portal *p, const char *interface) {
 static void update_interruption(struct uu_portal *p) {
     if (p->stream_started && p->state == PW_STREAM_STATE_PAUSED && !p->format_valid) {
         if (!p->interrupted_at) p->interrupted_at = g_get_monotonic_time();
-    } else p->interrupted_at = 0;
+    } else {
+        if (p->interrupted_at) p->display_changed_at = 0;
+        p->interrupted_at = 0;
+    }
+}
+static void monitors_changed(GDBusConnection *bus, const gchar *sender, const gchar *path,
+    const gchar *interface, const gchar *signal, GVariant *parameters, gpointer data) {
+    struct uu_portal *p = data;
+    pthread_mutex_lock(&p->lock);
+    /* #27: measured NULL format 10-36ms after the notification, with PAUSED
+     * 5-9ms before it. Preserve a correlated signal across repeated changes. */
+    if (p->stream_started && (!p->display_changed_at || !p->interrupted_at ||
+        p->display_changed_at < p->interrupted_at - PORTAL_DISPLAY_GRACE))
+        p->display_changed_at = g_get_monotonic_time();
+    pthread_mutex_unlock(&p->lock);
 }
 static void stream_state(void *data, enum pw_stream_state old, enum pw_stream_state state, const char *message) {
     (void)old; (void)message;
     struct uu_portal *p = data;
     pthread_mutex_lock(&p->lock);
     p->state = state;
-    if (state == PW_STREAM_STATE_STREAMING) p->stream_started = 1;
+    if (state == PW_STREAM_STATE_STREAMING) {
+        p->stream_started = 1;
+        p->display_changed_at = 0;
+    }
     update_interruption(p);
     pthread_mutex_unlock(&p->lock);
     if (state == PW_STREAM_STATE_ERROR || state == PW_STREAM_STATE_UNCONNECTED) fail(data, 9);
@@ -213,6 +233,7 @@ static void stream_format(void *data, uint32_t id, const struct spa_pod *param)
     p->width = info.size.width; p->height = info.size.height; p->format = info.format;
     p->frame_length = length;
     p->format_valid = 1;
+    p->display_changed_at = 0;
     update_interruption(p);
     pthread_cond_broadcast(&p->changed);
     pthread_mutex_unlock(&p->lock);
@@ -444,6 +465,9 @@ static struct uu_portal *open_persist(const char *restore, void (*save_token)(co
     g_variant_unref(results);
     if (*error) goto done;
     p->main_loop = g_main_loop_new(p->context, FALSE);
+    if (p->absolute_stream_pixels)
+        p->display_subscription = g_dbus_connection_signal_subscribe(p->bus, MUTTER_DISPLAY, MUTTER_DISPLAY,
+            "MonitorsChanged", MUTTER_DISPLAY_PATH, NULL, G_DBUS_SIGNAL_FLAGS_NONE, monitors_changed, p, NULL);
     g_main_context_pop_thread_default(p->context);
     if (pthread_create(&p->dbus_thread, NULL, run_dbus, p)) {
         *error = 3; p->cancelled = NULL; p->cancel_data = NULL;
@@ -496,13 +520,16 @@ void uu_portal_close(struct uu_portal *p) {
     if (p->clipboard_owner_subscription) g_dbus_connection_signal_unsubscribe(p->bus,p->clipboard_owner_subscription);
     if (p->clipboard_transfer_subscription) g_dbus_connection_signal_unsubscribe(p->bus,p->clipboard_transfer_subscription);
     if (p->closed_subscription) g_dbus_connection_signal_unsubscribe(p->bus, p->closed_subscription);
+    if (p->display_subscription) g_dbus_connection_signal_unsubscribe(p->bus, p->display_subscription);
     if (p->dbus_started) {
         /* A queued source also handles close racing with thread startup; an
          * immediate g_main_loop_quit before run would leave run blocking. */
         GSource *quit = g_idle_source_new();
         g_source_set_callback(quit, quit_dbus_loop, p->main_loop, NULL);
         g_source_attach(quit, p->context); g_source_unref(quit);
         g_main_context_wakeup(p->context); pthread_join(p->dbus_thread, NULL);
+    } else if (p->context) {
+        while (g_main_context_iteration(p->context, FALSE)) {}
     }
     if (p->pw_loop) pw_thread_loop_stop(p->pw_loop);
     if (p->stream) pw_stream_destroy(p->stream);
@@ -531,8 +558,11 @@ int uu_portal_capture(struct uu_portal *p, uint8_t *frame, size_t length) {
  * paused, format-less renegotiation observed after a live scaling change (#25).
  * Called with the frame mutex held; recovery runs on the Rust capture worker. */
 static int stream_stalled(struct uu_portal *p) {
-    return p->state == PW_STREAM_STATE_PAUSED && !p->format_valid && p->interrupted_at &&
-        g_get_monotonic_time() - p->interrupted_at >= PORTAL_RENEGOTIATION_TIMEOUT;
+    if (p->state != PW_STREAM_STATE_PAUSED || p->format_valid || !p->interrupted_at) return 0;
+    gint64 now = g_get_monotonic_time();
+    return now - p->interrupted_at >= PORTAL_RENEGOTIATION_TIMEOUT ||
+        (p->display_changed_at && p->display_changed_at >= p->interrupted_at - PORTAL_DISPLAY_GRACE &&
+         now - MAX(p->interrupted_at, p->display_changed_at) >= PORTAL_DISPLAY_GRACE);
 }
 int uu_portal_capture_since(struct uu_portal *p, uint8_t *frame, size_t length, uint64_t *revision) {
     if (!p || !frame) return 11;