Skip to content

perf(ui): bound session memory across workspaces - #626

Open
pascalandr wants to merge 2 commits into
NeuralNomadsAI:devfrom
pascalandr:perf/session-memory-eviction
Open

perf(ui): bound session memory across workspaces#626
pascalandr wants to merge 2 commits into
NeuralNomadsAI:devfrom
pascalandr:perf/session-memory-eviction

Conversation

@pascalandr

Copy link
Copy Markdown
Contributor

Summary

  • persist completed session history in a paged, budgeted IndexedDB cache and restore it without blocking authoritative HTTP/SSE state
  • enforce a global byte-aware resident-session budget across workspaces and subagents while preserving visible, active, and input-blocked work
  • bound tool, diff, diagnostics, speech, search, and nested-task rendering, with lazy full-copy paths for oversized content
  • fence cache writes, reverts, deltas, deletions, and instance reopen work against stale asynchronous state

Validation

  • 406 UI tests pass after rebasing onto current dev
  • UI TypeScript typecheck passes
  • production UI build passes
  • independent memory, cache-concurrency, and rendering gatekeepers returned SHIP

Discord report: https://discord.com/channels/1391832426048651334/1458412028325793887/1527610433844805673

Closes #553

Keep inactive workspace and subagent histories in a paged IndexedDB cache while enforcing a global byte-aware hot-memory budget. Visible, streaming, compacting, and input-blocked sessions retain authority, and nested task sessions load only while their owning pane is visible.

Bound tool, diff, diagnostic, speech, search, and nested-task rendering so large outputs cannot create unbounded parser or DOM work. Full large inputs and patches remain available through lazy copy actions.

Harden HTTP, SSE, revert, deletion, and instance-reopen ordering with revision and lifecycle fences. Coalesce cache writes without retaining queued snapshots, preserve orphan deltas through an authoritative reload, and version cache invalidations across shared workspace keys.

Validated with 395 UI tests, UI typecheck, production build, and independent memory, cache-concurrency, and rendering gatekeepers.

@pascalandr pascalandr left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gatekeeper verdict: DO NOT SHIP

Reviewed commit dd3b818e against upstream/dev. The platform tests and builds pass, but the following correctness and memory-safety blockers remain.

  1. P1: normal workspace refreshes invalidate lifecycle fences. instances.ts:353-356,765-766,927-933 replaces every existing Instance object during SSE reconnect reconciliation. Identity checks in session-api.ts:926, session-actions.ts:468,494, and session-message-cache.ts:134,290 then treat an unchanged workspace as removed/reopened. A successful server delete can skip local cleanup, and valid queued cache work can be dropped. Use a stable occurrence token rather than object identity, and change it only on actual remove/reopen.

  2. P1: an idle session can evict active optimistic work. session-memory.ts:23-29,52-64 checks only pending input when metadata says idle, ignoring normalized sending and streaming records. Send a prompt, switch away before the working-status event, then trigger memory pressure: the optimistic transcript is cleared while the request remains active. Protect all active normalized work regardless of metadata status.

  3. P1: orphan deltas are unbounded and recovery can lose them. delta-buffer.ts:27-32,108-115 re-holds failed deltas with no byte cap, entry cap, TTL, timer, or budget accounting. session-api.ts:1259-1264,1307-1310 clears held deltas before the replacement HTTP load succeeds; failure/cancellation loses streamed output. The ||= also stops evaluating later cleanup calls once true. Replace retained payloads with a bounded dirty-session marker plus a coalesced authoritative reload, or retain deltas until successful reconciliation.

  4. P1: same-ID reopen reuses stale HTTP and SSE authority. session-state.ts:157-170,1217-1253 deletes search authority so request IDs restart; an old search can match the replacement instance and write old sessions through session-api.ts:595-646. sse-manager.ts:164-170 gates only on current ID membership, so buffered events from the old occurrence are accepted after reopen, while valid startup events are dropped before the instance list loads. Fence both transports with one stable instance-occurrence generation.

  5. P1: persistent cache metadata misrepresents truncated history and can restore stale snapshots. session-message-cache.ts:94-121 first truncates to an approximately 8 MiB suffix. lib/session-message-cache.ts:212-260 receives only that suffix and records startIndex: 0, totalCount: suffix.length, and complete: true. If a delayed writer later skips because the store was evicted or snapshotSession returns null (session-message-cache.ts:124-140), the older IndexedDB snapshot is left eligible for restore. Preserve original total/start metadata and explicitly invalidate obsolete snapshots when a replacement cannot be written.

  6. P1: large structured tool output becomes inaccessible. tool-call/utils.ts:161-171 replaces oversized non-string output for rendering and returns null for copy. tool-call/renderers/default.tsx:8-27 consequently removes the copy action, and long errors are truncated without recovery in tool-call.tsx:483-489. This contradicts the full-copy behavior described by the PR. Keep full content behind a lazy copy path without eagerly stringifying it.

  7. P2: cache fencing maps grow for process lifetime. session-message-cache.ts:296-325,345-359 retains generation/version keys for every invalidated or destroyed session. This is monotonic memory growth in the feature intended to bound memory. Delete settled keys or scope generations to an instance occurrence.

Validation

  • All application CI tests and cross-platform builds passed.
  • git diff --check passed and the worktree is clean.
  • The red comment check is unrelated: the artifact-comment workflow received a GitHub API 504 while paginating runs.
  • Missing coverage: actual IndexedDB transactions/failures, idle-plus-sending eviction, orphan-delta bounds/recovery failure, same-ID reopen, and cache/HTTP reconciliation.

The authenticated account owns this PR, so GitHub does not permit a self-authored REQUEST_CHANGES; this COMMENT review carries the gatekeeper rejection.

Fence asynchronous session, workspace, permission, question, command, and stream updates to the runtime that started them. Rotate stream authority across workspace restarts and preserve connecting transitions through the Tauri transport.

Make persistent message restoration fail closed across incomplete reads, duplicate workspace owners, mutations, invalidation failures, and destructive client-state retries. Bound delta recovery, derived caches, search, diagnostics, task scans, and tool-output formatting by retained memory and CPU work.

Preserve prompts and attachments across definite failures without replaying ambiguous destructive requests, and add focused UI, server, and Rust regressions. Validated with UI 435/435, Tauri 85/85, typechecks, and production build; the Windows worktree fixture remains branch-name dependent.

@pascalandr pascalandr left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gatekeeper verdict: SHIP

Re-reviewed the updated head at c10cab6 after the correction loop. The runtime-authority, persistent-cache, bounded-memory/search, prompt recovery, SSE stream identity, and Tauri coalescing findings are resolved. The final independent gate found no release blocker.

Validation:

  • UI: 435/435 tests passed
  • Tauri: 85/85 tests passed
  • UI, Electron, and server typechecks passed
  • UI production build passed
  • Server: 248 passed, 4 skipped; the only local failure is the unchanged Windows git-worktrees fixture that assumes the root branch is named main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: WebKit renderer memory grows to 2.5 GB over 24h+ runtime — tool outputs fully rendered with no eviction

1 participant