Skip to content

Sync: update loro to 1.17 so streaming chats stop leaking memory - #1116

Open
Gratenes wants to merge 1 commit into
zeronsh:mainfrom
Gratenes:fix/chat-sync-memory
Open

Gratenes wants to merge 1 commit into
zeronsh:mainfrom
Gratenes:fix/chat-sync-memory

Conversation

@Gratenes

@Gratenes Gratenes commented Oct 31, 2026 •

Copy link
Copy Markdown

Summary

  • Updates loro from 1.13.9 to 1.17.1. No code changes.
  • Before 1.17, every snapshot export left another copy of the doc's text in its string arena (fixed in loro-dev/loro 089a4e6). ChatPersistence exports a dirty chat once a second, so a chat with a streaming agent grows quadratically. A headless engine running 16 Pi agents reached 34 GB after six hours.
  • This is also the cause of the panic in Chat sync stays down after one Loro panic until the desktop process is restarted #905. Once the arena passes 4 GiB, export_snapshot fails append-only-bytes start <= end and the doc is poisoned. The retry loop after a poisoned lock is unchanged.

Refs #905

Test plan

  • cargo test -p zeron-doc -p zeron-sync -p zeron-client
  • cargo test -p zeron-engine --lib
  • Streaming 380 KB through SegmentWriter with an export per tick grows RSS 250 MB on 1.13.9 and a few MB on 1.17.1 (Linux)
  • Not run on macOS or Windows

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

loro before 1.17 left another copy of a doc's text in its string arena on
every snapshot export (fixed in loro-dev/loro 089a4e6). chat_persistence
exports a dirty chat each second, so a chat with a streaming agent grew
quadratically until the arena passed 4 GiB and export panicked (zeronsh#905).

@zeronsh zeronsh Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security and correctness review of 7e407623

Verdict: no confirmed merge-blocking security regression found in this dependency update. This is a targeted review of the two changed files and relevant import/persistence paths, not a full audit of Loro or the application.

Severity rubric Meaning Introduced findings
P0 — Critical Broad compromise, remote code execution, or systemic data exposure 0 confirmed
P1 — High Credible authorization bypass, serious data loss, or exploitable denial of service 0 confirmed
P2 — Medium Bounded security/correctness regression requiring remediation 0 confirmed
P3 — Low Minor regression 0 confirmed
Review area Assessment
Dependency integrity All four changed crate archives match the SHA-256 checksums in the PR lockfile; registry sources and dependency lists are unchanged. This verifies archive consistency, not publisher trust.
Availability The upstream fix uses block-owned text/list buffers instead of repeatedly appending decoded content to document-wide arenas. This addresses the repeated snapshot-export growth exercised by ChatPersistence.
Input handling Upstream includes malformed-import rollback, peer-ID collision checks, and deep-value stack-safety fixes. Four truncated-snapshot smoke cases against 1.17.1 rejected the input, preserved existing state, and allowed subsequent edits/exports.
Authorization / confidentiality No application authorization, tenant routing, secrets handling, or transport changes in this diff.
Compatibility Bidirectional 1.13.7 ↔ 1.17.1 WASM smoke tests passed for full snapshots, concurrent incremental updates, movable-list moves, and shallow snapshots. This is sampled coverage, not proof of all historical-document compatibility.
Advisories No Loro matches in the RustSec advisory database checked at 2295466c. This was a targeted lookup, not a full-lockfile cargo audit.
flowchart LR
  A["Cargo.toml: loro 1.13 → 1.17"] --> B["loro 1.13.9 → 1.17.1"]
  B --> C["loro-internal 1.13.9 → 1.17.1"]
  B --> D["loro-common 1.13.1 → 1.17.0"]
  C --> D
  C --> E["loro-kv-store 1.13.9 → 1.17.0"]
Loading
flowchart LR
  P["Authorized peers"] --> R["ChatRoom: authenticated byte relay"]
  R --> I["Native import: Loro 1.17.1"]
  I --> D["SessionDoc"]
  D --> S["ChatPersistence: dirty snapshot export"]
  S --> L["Local snapshot store"]
  P --> W["Legacy / workspace SessionRoom"]
  W --> O["Edge WASM: Loro 1.13.7 unchanged"]
Loading

Non-blocking follow-ups

  • Edge coverage: edge/package-lock.json still pins loro-crdt 1.13.7, and SessionRoom imports remote/persisted CRDT bytes. Those rooms do not receive the upstream availability and decoder fixes from this Rust-only bump. Track an edge upgrade separately; this is pre-existing exposure, not a regression introduced here.
  • Regression coverage: retain a repeat-export streaming/memory check and mixed-version persistence fixtures in CI. Upstream has explicit arena-growth regression tests; I inspected them but did not run that upstream suite. The existing poisoned-lock retry behavior is unchanged.

Validation performed: cargo test --locked -p zeron-doc -p zeron-sync -p zeron-client passed on Linux: 304 passed, 0 failed, 2 ignored (the ignored cases require a live edge). The four archive checks and WASM compatibility/malformed-input smoke tests above also passed. I did not rerun the engine suite, the author's RSS benchmark, or macOS/Windows tests; those remain outside this review's independent validation.

Upstream evidence: 1.17.1 release changelog, block-owned decode buffers.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant