Skip to content

fix(cursor): state the T04 re-arming contract on an injectable clock - #5131

Merged
lidge-jun merged 1 commit into
devfrom
codex/fix-cursor-stream-health-clock-seam
Sep 19, 2026
Merged

lidge-jun merged 1 commit into
devfrom
codex/fix-cursor-stream-health-clock-seam

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Sep 19, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • tests/providers/cursor/cursor-stream-health.test.ts fails in the unsharded macos control lane with Cursor stream stalled: no inbound frames for 5s before turnEnded, raised from src/adapters/cursor/live-transport.ts. This is a different kind of failure from the four in [Bug]: four tests exceed their timeouts in the unsharded macOS control lane, first observed once the lane was able to finish #4997: it is not a test exceeding its own timeout, it is production code declaring a stall. The case at issue asserts the opposite of a stall — that meaningful frames keep resetting both deadlines — and the only way it could state that against real timers was to run a synthetic HTTP/2 server for three times the silence budget and require that no deadline expire. That is an assertion about how busy the machine is, and under control-lane contention the fixture's own frames do not arrive inside the window while the watchdog is behaving exactly as designed.
  • The budget was already CI-scaled once, in fix(test): preserve lane output after timeouts and stabilize the Cursor stream-health watchdog #3940, and scaling is what makes this worse rather than better. isolationBudgetMs(1_000) floors the silence budget at 5s on CI, and the progress span is three times that, so the case now needs 15 consecutive seconds during which no 5s gap in frame decoding occurs, inside a 22s fixture window. Raising the floor lengthens the exposure. There is no value of the budget that makes a "no deadline expired" assertion safe on a runner that can pause for more than ten seconds.
  • So the fix removes real elapsed time from the contract instead of negotiating with it. CursorTransportFactoryInput gains an optional streamHealthClock with now, setTimeout and clearTimeout, and only the T04 watchdog reads time or schedules through it. The test supplies a clock it advances by hand between frames whose arrival it awaits, so contention can delay a frame — which the test waits for — without any deadline passing. It also now asserts two things the old case could not: exactly one timer is armed after every frame, so re-arming replaces the previous deadline rather than stacking one, and no deadline survives turnEnded, read before close() which would clear it anyway.
  • The seam follows StageLeaseClock in src/cli/account-api.ts, which is the same shape for the same reason and is documented as test-only. Production omits it and binds all three primitives to the globals through one module constant.

What did not change

The production budgets are untouched: CURSOR_STREAM_SILENCE_FAIL_MS is still 30s and CURSOR_STREAM_HEARTBEAT_ONLY_FAIL_MS still 90s. Five seconds was never the production contract — it was the CI-scaled test budget being printed back by the stall message. The first-frame timer, the turn-ended close grace, the client-tool finalize grace and the outbound heartbeat all stay on the global timers; the seam is scoped to the two T04 clocks and the one T04 timer. The elapsedMs diagnostic deliberately stays on the wall clock because turnStartedAt is stamped there and the pair has to subtract coherently.

No timeout was widened, no retry added, no platform skipped and no failure masked.

Real-timer coverage that remains

Three cases in the same file still drive the watchdog through the default global-timer clock, and all three assert that something happens rather than that it does not, so a slow runner cannot turn them red for being slow:

  • silence after the first frame fails the turn, on a 300ms budget — a real setTimeout reaching failAndClear.
  • heartbeat-only traffic survives the silence threshold and fails at the heartbeat-only threshold — real re-arming, 40ms pings repeatedly resetting the silence clock through the default clock.
  • turnEnded disarms the watchdog even when the server holds the stream open, on a 300ms budget — real cancellation.

I deliberately did not add a short real-timer version of the re-arming case. It would be the same flake with a smaller window: any assertion that a deadline did not expire is vulnerable to a stall longer than the deadline, and the observed stalls on that lane exceed ten seconds.

Structure ownership

structure/providers/cursor.md owns src/adapters/ and had no section on the stream-health watchdog, so the new contract is recorded there: the 30s and 90s budgets are production and a test does not move them, the injectable clock is scoped to T04 alone, and the firing half of the contract stays on real timers. That is the invariant the previous round of this flake violated by scaling the budget, so it is worth being written down rather than inferred from the seam's existence.

Verification

  • No local suite was run. This repository verifies on hosted CI, and a past local run destroyed real user state under ~/.opencodex. The change was reviewed by hand-executing the new case against the transport's decode path and by static reasoning about which timers moved.
  • Hand-execution of the new case, with virtual budgets S=1000 and 2S=2000: part-0 lands at virtual 0 and arms one timer for 1000; the clock is advanced to 900, where nothing is due, and part-1 re-arms for 1900; advanced to 1800, part-2 re-arms for 2800; advanced to 2700, part-3 re-arms for 3700. Virtual time ends at 2700, past both non-resetting deadlines (1000 and 2000), which is the claim. No pending deadline is ever due at an advanceTo target, so no watchdog callback runs at all.
  • Hosted CI on the exact head is the evidence for this PR; test 1/4..4/4 and macos 1/2..2/2 carry both changed files.
  • Honest limit: the lane this fixes is the 30-minute-budget macos control job from [Bug]: four tests exceed their timeouts in the unsharded macOS control lane, first observed once the lane was able to finish #4997, which cannot finish inside its own budget, so CI here cannot demonstrate the original failure disappearing. What it can show is that the rewritten case and the untouched production path stay green on the sharded lanes. The argument for the fix is structural — the assertion no longer depends on elapsed real time — not a green run.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Refs #4997

@lidge-jun
lidge-jun requested a review from Ingwannu as a code owner September 19, 2026 08:13
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-19T08:15:31.681936Z 895a457 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: lidge-jun/opencodex/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 7cea96db-b570-49f2-8420-4e8c35a2ba71

📥 Commits

Reviewing files that changed from the base of the PR and between 4e7d713 and 895a457.

📒 Files selected for processing (3)
  • src/adapters/cursor/live-transport.ts
  • src/adapters/cursor/transport.ts
  • tests/providers/cursor/cursor-stream-health.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The T04 stream-health watchdog now uses an injectable clock. Production uses wrappers around global time APIs. Tests use a manual virtual clock to verify timer re-arming, deadline resets, clean completion, and watchdog disarming.

Changes

Stream-health clock abstraction

Layer / File(s) Summary
Clock contract and transport wiring
src/adapters/cursor/transport.ts, src/adapters/cursor/live-transport.ts
CursorStreamHealthClock defines now, setTimeout, and clearTimeout. CursorTransportFactoryInput accepts the optional test clock. LiveCursorTransport defaults to REAL_STREAM_HEALTH_CLOCK, which delegates to the global time APIs.
Watchdog clock usage and virtual-time validation
src/adapters/cursor/live-transport.ts, tests/providers/cursor/cursor-stream-health.test.ts
Watchdog timers and inbound-frame timestamps use the selected clock. The stream-health test replaces interval-based timing with manual time advancement and verifies timer re-arming, clean turnEnded completion, disarming, and the absence of later failure.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Refactor

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 3 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: it introduces an injectable clock for the Cursor T04 stream-health re-arming contract. It is concise and specific.
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lidge-jun

Copy link
Copy Markdown
Owner Author

리뷰 · 우선순위 77 / 80

macOS control 레인에서 cursor-stream-health 테스트가 Cursor stream stalled: no inbound frames for 5s로 빨간불이 납니다. 워치독이 고장 난 게 아니라, “의미 있는 프레임이 오면 데드라인 두 개가 다시 잡힌다”는 계약을 실제 시계로 증명하려다 생긴 일입니다. 예전 케이스는 침묵 예산의 세 배 동안 합성 HTTP/2 서버가 프레임을 계속 보내고, 그동안 데드라인이 한 번도 만료되지 않기를 요구했습니다. 그 주장은 “워치독이 맞다”가 아니라 “러너가 바빠서 프레임 도착이 5초 이상 밀리지 않는다”에 가깝습니다. CI에서 isolationBudgetMs(1_000)가 침묵 예산을 5초로 올리면 노출 구간이 15초로 길어져서, 예산을 키울수록 더 깨지기 쉽습니다. #4997의 타임아웃 초과와는 종류가 다릅니다.

이 PR은 예산을 더 키우지 않고, T04 워치독만 쓰는 시간을 주입 가능하게 바꿉니다. CursorTransportFactoryInput에 streamHealthClock(now / setTimeout / clearTimeout)을 두고, live-transport.ts의 T04 타이머·시계만 그걸 탑니다. 프로덕션은 REAL_STREAM_HEALTH_CLOCK으로 글로벌을 묶고, 30초·90초 예산과 first-frame / turnEnded grace / outbound heartbeat는 그대로입니다. elapsedMs 진단만 의도적으로 Date.now()에 남깁니다. turnStartedAt이 벽시계로 찍히므로 빼기 짝이 맞아야 하기 때문입니다. 패턴은 StageLeaseClock과 같습니다.

테스트는 manualStreamHealthClock으로 가상 시간을 프레임 사이에만 진행합니다. part-0~3이 900·1800·2700에 도착해 S와 2S를 모두 지나도 데드라인은 매번 최신 프레임 기준으로 다시 잡힙니다. 추가로 (1) 프레임마다 armed 타이머가 정확히 1개(쌓이지 않고 교체) (2) turnEnded 뒤·close() 전에 armed가 0인지까지 고정합니다. 같은 파일의 침묵 실패·heartbeat-only·turnEnded 해제 세 케이스는 여전히 실타이머로 “뭔가 일어난다”만 검사해서, 느린 러너가 빨간불을 만들 수 없습니다. “만료되지 않았다”의 짧은 실타이머 버전을 일부러 안 넣은 것도 맞습니다. 그 형태는 창만 작아진 같은 플레이크입니다.

경로 src/adapters/cursor/transport.ts - CursorStreamHealthClock + 테스트 전용 streamHealthClock 옵션. 프로덕션은 omit
경로 src/adapters/cursor/live-transport.ts - T04 arm/clear/noteInboundFrame/첫 프레임 시드만 시계 주입. elapsedMs는 벽시계 유지
경로 tests/.../cursor-stream-health.test.ts - 가상 시계로 re-arm 계약 + 단일 타이머 + turnEnded 해제. 실타이머 “긍정 실패” 세 케이스는 유지
예산 - CURSOR_STREAM_SILENCE_FAIL_MS 30s / heartbeat-only 90s 불변. 타임아웃 확대·재시도·플랫폼 스킵 없음
한계(정직) - macOS control 레인이 자체 예산 안에 못 끝나서, 이 PR CI만으로 “원래 플레이크가 사라졌다”를 숫자로 증명하긴 어렵다. 증거는 구조(실시간에 의존하지 않음)다

메인테이너의 판단이 필요한 지점

너의 추천
샤드 CI가 그린이면 merge한다. “안 만료됐다”를 머신 바쁨에 맡기던 계약을 가상 시계로 옮긴 올바른 수정이고, 프로덕션 예산·다른 타이머는 건드리지 않았다. types/config 스플릿에 무효화될 PR이 아니다. Refs #4997.

이 댓글은 grok-bot이 작성했습니다

@github-actions

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 895a457e61

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

* watchdog was working correctly. With the clock injected, virtual time advances only where
* the test says it does, so no amount of runner contention can expire a deadline.
*/
streamHealthClock?: CursorStreamHealthClock;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Update the owned adapter documentation for the clock seam

This adds an exported streamHealthClock contract and reroutes the Cursor watchdog through it, changing an owned src/adapters/ transport surface, but the commit updates no corresponding structure/ document. This leaves the repository's architecture source of truth stale; update the applicable documents listed for src/adapters/ in structure/INDEX.md in the same change, including the Cursor transport/watchdog contract.

AGENTS.md reference: src/AGENTS.md:L11-L11

Useful? React with 👍 / 👎.

tests/providers/cursor/cursor-stream-health.test.ts asserted that meaningful
frames keep resetting both stream-health deadlines by running a synthetic
HTTP/2 server for three times the silence budget and requiring that no
deadline expire. That is an assertion about how busy the machine is. In the
unsharded macOS control lane it fails with "Cursor stream stalled: no inbound
frames for 5s before turnEnded" while the watchdog is behaving correctly:
under contention the fixture's own frames do not arrive inside the window.

The budget is already CI-scaled, and scaling is what makes it worse. With the
silence budget floored at 5s the case needs 15 consecutive seconds during
which no 5s gap in frame decoding occurs, so raising the floor lengthens the
exposure rather than reducing it.

Add a T04-only clock seam (now, setTimeout, clearTimeout), matching
StageLeaseClock in src/cli/account-api.ts. Production omits it and keeps the
globals; the 30s and 90s budgets, the first-frame timer, the turn-ended grace
and the outbound heartbeat are untouched. The test now advances virtual time
by hand between frames whose arrival it awaits, so runner contention can
delay a frame without any deadline passing, and it additionally asserts that
exactly one timer is armed after each frame and none survives turnEnded.

No production timeout changed, no budget widened, no retry added and no
platform skipped.
@lidge-jun

Copy link
Copy Markdown
Owner Author

추가 리뷰 · 우선순위 78 / 80

이전 리뷰(77/80, 895a457e) 이후 force-push로 tip이 072271a6이 됐습니다. 코드 세 파일 blob은 그대로입니다(live-transport.ts / transport.ts / cursor-stream-health.test.ts SHA 불변). 새로 들어온 건 structure/providers/cursor.md에 Inbound stream-health clock ownership 절 +25줄뿐입니다. 통계만 보면 +161 → +186입니다.

이 절은 이전 리뷰가 말로만 적었던 계약을 구조 문서에 고정합니다. (1) 프로덕션 예산은 30s/90s이고 테스트가 그걸 움직이지 않는다 (2) streamHealthClock은 T04만 바꾸고 first-frame·turnEnded grace·client-tool finalize·outbound heartbeat·elapsedMs는 벽시계/글로벌에 남긴다 (3) re-arm 쪽은 실타이머로 “안 만료”를 말하면 머신 바쁨 주장과 섞이므로 가상 시계가 필요하고, 발화(침묵 실패·heartbeat-only·turnEnded 해제) 쪽은 실타이머로 남긴다. 코드와 문서가 같은 이야기를 합니다.

경로 structure/providers/cursor.md - T04 시계 소유권·범위·왜 시임이 필요한지 문서화. 코드 변경 없음
이전 tip 895a457e → 072271a6 - 동일 부모 4e45e1770f11, 메시지 동일, 문서만 추가

메인테이너의 판단이 필요한 지점

  • 문서 추가만으로 tip이 바뀌었으니, 이전 리뷰 뒤 돌린 CI를 새 tip 기준으로 다시 보면 된다
  • 구조 문서에 시임을 남긴 선택이 앞으로 다른 타이머까지 넓히라는 신호로 읽히지 않게, PR 본문·이 절의 “T04 alone”을 그대로 유지할지

너의 추천
이전 추천 유지: 샤드 CI 그린이면 merge. 코드 계약은 그대로고, 문서가 그걸 잠근 정도라 점수는 78로 올린다. types/config 스플릿과 무관. Refs #4997.

이 댓글은 grok-bot이 작성했습니다

@github-actions github-actions Bot added the bug Something isn't working label Sep 19, 2026
@lidge-jun
lidge-jun merged commit e668aa1 into dev Sep 19, 2026
30 of 32 checks passed
@lidge-jun
lidge-jun deleted the codex/fix-cursor-stream-health-clock-seam branch September 19, 2026 09:15
lidge-jun added a commit that referenced this pull request Sep 19, 2026
… clock (#5169)

* test(cursor): state the T04 heartbeat-only contract on the injectable clock

The heartbeat-only case was the last T04 assertion that still contained a term for
elapsed real time. It kept a real 40ms interval writing liveness frames and required
no decoded-frame gap longer than the silence budget for the whole heartbeat-only
window, so a runner that paused longer than one budget made the SILENCE watchdog win
while both watchdogs behaved correctly. #3940 had already scaled that budget once;
scaling it again only lengthens the exposure.

Move the case onto the streamHealthClock seam #5131 added, advancing virtual time by
hand between liveness frames whose arrival it awaits. Production budgets (30s silence,
90s heartbeat-only), the first-frame timer and the outbound heartbeat are untouched,
and the silence watchdog's real-timer firing case is unchanged.

* test(cursor): pin the T04 deadline selection that no real-timer case can state

Adversarial review of the previous commit found two gaps it had left.

The min(silence, progress) rule was not red-capable anywhere. A watchdog that read only
the progress deadline would relax production silence detection from 30s to 90s, and every
real-timer case would stay green: a later deadline still produces the same no-inbound-frames
message inside the case timeout. Add a virtual-clock case that requires the turn to fail at
exactly S with nothing left armed, which that mutation fails immediately.

Moving the heartbeat-only case onto the seam also removed the only coverage of the progress
branch firing through the production default clock, which structure/providers/cursor.md
claimed was still there. Restore it with a real-timer case whose silence budget is two
orders of magnitude beyond its progress budget, so load can make it later but never wrong,
and correct the document to say which half of the contract lives where and why.

* test(cursor): bound the manual stream-health clock and pin the stall prefix

Third review round. Widening the manual clock from one case to three made its
unbounded drain loop worth fixing: a watchdog that re-armed an already expired
deadline would schedule a zero-delay timer forever, and because advanceTo is
synchronous, Bun's per-test timeout could never interrupt it. That mutation would
wedge the lane instead of reddening one case, so the loop now throws after 1000
callbacks and names the cause.

Also assert the shared Cursor stream stalled prefix, which no case pinned, and
correct the real-timer comment: ordinary load delays that case, and a pause long
enough to also cross the 60s silence budget has already blown the case's own 15s
limit, which is why its outcome cannot change rather than merely being unlikely to.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant