Skip to content

test(sessions): add Session/Memory/Summary replay consistency harness#158

Open
quan020406 wants to merge 2 commits into
trpc-group:mainfrom
quan020406:issue-89
Open

test(sessions): add Session/Memory/Summary replay consistency harness#158
quan020406 wants to merge 2 commits into
trpc-group:mainfrom
quan020406:issue-89

Conversation

@quan020406

Copy link
Copy Markdown

Related Issues

Resolved #89

Summary

Adds a replay consistency harness for Session, Memory, and Summary behavior across InMemory and SQLite backends.

The harness loads 10 public replay cases, applies identical operation sequences to each backend, normalizes non-business differences, compares SDK-observable state, and writes a structured diff report.

Changes

  • Add 10 public replay cases covering single-turn, multi-turn, tool calls, state updates, memory store/search, summary generation/update, summary truncation, duplicate write recovery, and wrong-session isolation.
  • Add tests/sessions/test_replay_consistency.py for InMemory vs SQLite replay comparison.
  • Add mutation checks for injected event, state, memory, and summary inconsistencies.
  • Generate session_memory_summary_diff_report.json with case id, backend pair, field path, session id, event index or summary id, and compared values.
  • Add a 150-300 word design note explaining normalization, summary comparison, allowed differences, and backend integration.

Results

  • pytest tests\sessions\test_replay_consistency.py -q
  • Result: 4 passed, 1 skipped
  • Redis integration is gated by TRPC_AGENT_REPLAY_REDIS_URL and skipped when unset.
  • Normal replay cases report 0 diffs.
  • Injected inconsistency checks detect all configured mutations.
  • Summary lost, summary overwrite error, and summary wrong-session ownership are all detected.

Notes

The current SDK does not expose a dedicated persisted summary.version field. This harness interprets summary version as observable revision state, validated through generation order, replacement behavior, persisted summary text, session ownership, timestamp validity, and coverage/projection consistency.

Backward Compatibility

No production SDK API changes. All changes are limited to tests, fixtures, report output, and design documentation.

RELEASE NOTES: NONE

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@4336564). Learn more about missing BASE report.

Additional details and impacted files
@@            Coverage Diff             @@
##             main        #158   +/-   ##
==========================================
  Coverage        ?   87.55867%           
==========================================
  Files           ?         467           
  Lines           ?       44103           
  Branches        ?           0           
==========================================
  Hits            ?       38616           
  Misses          ?        5487           
  Partials        ?           0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@quan020406

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

Rook1ex added a commit to trpc-group/cla-database that referenced this pull request Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

构建 Session / Memory 多后端回放一致性测试框架

1 participant