Skip to content

fix(portal): 修复 iPhone 锁屏冷连接无法自动解锁的问题 - #34

Merged
VallierDev merged 2 commits into
VallierDev:mainfrom
wanglz111:codex/ios-locked-bootstrap
Oct 17, 2026
Merged

VallierDev merged 2 commits into
VallierDev:mainfrom
wanglz111:codex/ios-locked-bootstrap

Conversation

@wanglz111

Copy link
Copy Markdown
Contributor

Refs #29

GNOME 锁屏且 Portal 尚未建立时,官方 iPhone 客户端虽然完成 CONTROL Echo/查询并收到锁屏状态,却不发送自动解锁请求(SimpleAction 15)。现在仅在 Portal 等待就绪、确认锁屏且连接已建立时,通过已协商的视频轨每秒发送一帧生成的黑画面;解锁后由真实视频接续同一条 RTP 时间线。默认启用,无需实验开关。

采集权限、真实屏幕信息和输入仍按 Portal 实际状态处理。编码和发送有超时;采集就绪、拒绝或连接/SCTP 关闭时退出。黑帧使用所选 codec 对应的 FFmpeg libx264/libx265 软件编码器,缺失时明确记录错误。原有锁屏策略及独立文件传输、终端和端口映射连接保持原有行为。

实验结果

2026 年 10 月 17 日,使用同一实验二进制,仅切换黑帧开关;每轮均在 GNOME 锁屏时重启服务,并由 iPhone 首先发起桌面连接,权限报告保持真实状态:

对照 结果
开启黑帧 收到 action 15,自动解锁并显示桌面
关闭黑帧 无黑帧、无 action 15,未能自动解锁;关闭连接后的本地解锁不计成功
再次开启 再次收到 action 15,自动解锁并显示桌面

移除开关后的默认版本也完成一次冷连接验收:11:26:13 发出 H.265 黑帧,11:26:17 收到 action 15,11:26:18 自动解锁成功,随后恢复真实视频;用户确认手机正常显示桌面。详细时间线、二进制哈希及结果解释见 docs/experiments/issue-29-locked-video.md。

验证范围和限制

  • 实机覆盖本次 Ubuntu GNOME Wayland/Portal 主机、官方 iPhone 4.42.0 和 H.265;该结果不推断 iOS 内部实现,也不证明所有连接只需一帧。
  • H.264/H.265 均有黑帧解码和虚拟网络 RTP 时间线验证;H.264 手机实机、其他客户端版本、其他主机/编码器组合、长时间等待和慢速 WAN/中继尚未验收。
  • 自动化切换测试在 run() 退出后直接向视频轨发送样本;生产 stream_desktop 切换由上述 H.265 实机结果覆盖。
  • 黑帧为合成画面,仍需启用客户端自动解锁并使用 Linux 登录密码;首次或被撤销的 Portal 授权仍需本机确认。不提供真实 GNOME 锁屏画面、锁屏原生键鼠操作或整机挂起/休眠唤醒。

Review 和自动化验证

独立 review 发现编码完成与解锁/采集就绪同时发生时,旧锁屏状态可能允许多发一帧黑画面。已优先处理状态通知,并在编码前和发送前复查 Portal、锁屏与连接状态;新增定向回归,在编码最后一次 poll 中发布解锁、未知锁状态或采集就绪,验证不会发出黑帧。

review-agent 对修正复审结果为 No findings。该 review 后续修正已做自动化验证,未重新部署或再次进行手机实机验收;上面的实机记录对应此前已安装的默认版本。

  • cargo test --locked:183 passed,5 ignored。
  • cargo test --locked --features wayland,portal:191 passed,6 ignored。
  • cargo test --locked --features wayland,portal locked_video::tests -- --ignored --nocapture:显式通过 3 项 FFmpeg 解码及虚拟网络测试。
  • 新增并发状态回归单独运行通过,且包含在上述常规测试中。
  • cargo build --release --locked --features wayland,portal 及 git diff --check 通过。

@wanglz111

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T03:45:39.151614Z 2efdc76 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

…he repository

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@VallierDev

Copy link
Copy Markdown
Owner

非常漂亮的对照实验(开→关→开),坐实了 iPhone 依赖锁屏期间的视频帧。代码只作用于 Portal 等待+确认锁屏阶段,X11 路径不受影响,日志也只有计数,没问题。

我补了一个小提交(b12d1c7):项目约定实验/验收记录不放进仓库,所以移除了 docs/experiments/issue-29-locked-video.md,README 链接改指向 #29 和本 PR——你在 PR 描述里的时间线已经完整保留。CI 通过后合并,感谢!

Great ON/OFF/ON experiment. I pushed one small commit (b12d1c7) removing docs/experiments/ (experiment/acceptance records stay out of the repo by convention; your PR description keeps the full timeline) and pointing the README links to #29/#34. Will merge once CI passes.

@VallierDev
VallierDev merged commit cc01b3e into VallierDev:main Oct 17, 2026
3 checks passed
@VallierDev

Copy link
Copy Markdown
Owner

已合并,感谢!将随 v0.1.29 发布。/ Merged, thanks — ships in v0.1.29.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2efdc76503

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/locked_video.rs
Comment on lines +79 to +82
_ = updates.changed() => continue,
_ = locked.changed() => continue,
result = tokio::time::timeout(Duration::from_secs(2),
track.write_sample_with_extensions(*codec, &sample, &extensions)) => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Make the synthetic access-unit write cancellation-safe

When a lock/readiness notification arrives after a backpressured send has begun, either changed() arm wins and drops write_sample_with_extensions in the middle of an access unit (the two-second timeout can do the same). VideoTrack::write_sample_with_extensions advances the shared timestamp up front and the sequence before each awaited packet write (src/video_track.rs:64-70), so the handoff to live capture can start after a partial, unmarked IDR and with a sequence gap, causing loss/NACKs or delayed first video on exactly the slow path this timeout covers. Make this access-unit write cancellation-safe, or allow the in-flight frame to finish before reacting to state changes.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants