fix(responses): strip only citation spans, keep Codex App directives

[lidge-jun]

Summary

When a routed model answered with a Codex App visualize reference (U+E200 visualize U+E202 {"path":…} U+E201), the proxy deleted the whole line, so the app never rendered the visualization. The citation filter added for #3150 removed every U+E200 … U+E201 span without checking the keyword, and the Codex App uses the same delimiters for its own directives.

The filter now removes only cite and filecite spans. Every other directive passes through byte for byte, including private-use characters inside its payload, up to the existing 4,096-character span bound. The streaming filter is now a per-character state machine that withholds text only while a keyword could still become a citation, and stripCitationMarkers runs the same machine over the whole string, so output_text.done always matches the streamed deltas (#3843).

Closes #6039

Verification

• bun test tests/responses/citation-markers.test.ts: 23 pass. The 5 new Citation-marker filter strips Codex App visualize directives #6039 cases fail against the previous filter.
• Every test file that exercises the filter or its callers (tests/adapters/bridge.test.ts, tests/responses/protocol-direct-encoders-chat.test.ts, the Kiro metering and completion tests): 200 pass.
• bun run typecheck, bun run structure:check, and bun run privacy:scan pass.
• An independent review fuzzed streaming against whole-string output (about 250,000 comparisons over random chunkings, keyword prefixes, nested STARTs, and bound edges). It found one defect, a START inside a visualize payload being read as a new citation, which this PR fixes and covers with a regression test.
• bun run test:changed could not give a meaningful result locally because this checkout lives under ~/.codex. The suite's real Codex home guard refused temporary directory cleanup in unrelated server and integration tests (367 failures, all with that guard error). Full coverage is left to CI.

Checklist

• Scope stays focused and avoids unrelated cleanup.
• Docs or release notes were updated when needed.
• Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Summary by CodeRabbit

• Bug Fixes
  • Citation markers are now handled consistently whether text arrives all at once or in streaming chunks.
  • Only spans marked with cite or filecite are removed. Other directives, including visualize, remain visible.
  • Incomplete or malformed spans are preserved rather than dropped, and long spans are handled safely.
  • Streaming text is released promptly when a directive is determined not to be a citation.

[github-actions]

✅ Deterministic PR hygiene checks passed.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Repository: lidge-jun/opencodex/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 4e4261ef-0a95-41f2-a547-53da77944121

📥 Commits

Reviewing files that changed from the base of the PR and between d25f972 and d8b3245.

📒 Files selected for processing (2)

• src/responses/citation-markers.ts
• tests/responses/citation-markers.test.ts

 _____________________________________
< ICBM: Intercontinental Bug Missile. >
 -------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       / 　 づ

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-27T02:40:28.666597Z	d8b3245	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.