test(sessions): 新增 Session/Memory/Summary 多后端回放一致性测试框架#153
Open
guocfu wants to merge 1 commit into
Open
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #153 +/- ##
==========================================
Coverage ? 87.53692%
==========================================
Files ? 467
Lines ? 44018
Branches ? 0
==========================================
Hits ? 38532
Misses ? 5486
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅ |
Author
|
I have read the CLA Document and I hereby sign the CLA |
Author
|
recheck |
0b98617 to
c796fd0
Compare
Rook1ex
added a commit
to trpc-group/cla-database
that referenced
this pull request
Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Session / Memory / Summary Replay Consistency 设计说明
Description | 描述
本变更为 Session / Memory / Summary 后端新增 replay consistency harness。框架会用同一组确定性 JSONL replay case 驱动后端执行 Session、Memory、Summary 操作,读取后归一化为稳定 snapshot,并输出结构化 diff report,用于发现不同存储后端之间的事件顺序、state、memory、summary 行为漂移。
Closes #89
Files Changed
tests/sessions/replay_consistency/:模块化 replay consistency 框架,包括 case loader、fixture builder、normalizer、comparator、reporter、backend factory 和 assertions。tests/sessions/replay_consistency/replay_cases/:10 个 JSONL replay fixture,覆盖单轮文本、多轮追加、工具调用与响应、state 覆盖、memory 写入读取、summary 生成/更新、summary 截断、异常恢复、重复写入和分支 metadata。tests/sessions/test_replay_consistency.py:主 E2E replay 测试,覆盖后端矩阵、snapshot 归一化比较、summary 内容/metadata 检查、diff report fixture。tests/sessions/test_replay_mutations.py:真实 replay snapshot mutation 检测,验证事件、state、tool call、memory、summary 丢失/篡改/覆盖/归属错误能被检出。Backend Behavior | 后端行为
默认本地模式不要求外部 SQL / Redis,只运行 InMemory 轻量 replay。设置
TRPC_AGENT_REPLAY_SQL_URL后尝试外部 SQL 后端;设置TRPC_AGENT_REPLAY_REDIS_URL后尝试外部 Redis 后端。外部 SQL 不可用时降级到临时 SQLite,外部 Redis 不可用时降级到 mock Redis,避免本地或 CI 因缺少真实 MySQL/Redis 失败。Comparison Strategy | 比较策略
normalizer 会固定时间戳、自动生成 ID、序列化顺序等非业务字段,对事件内容、工具参数/响应、state 值、memory 文本/作用域、summary 文本和 summary 归属保持严格比较。Summary 比较区分内容语义和存储 metadata:摘要文本做规范化比较,
session_id、manager session、summary event count、compressed count 等 metadata 严格比较。真实 summary 路径使用 deterministic fake summarizer,避免真实 LLM 随机性。Reports | 报告
已提交示例报告:
tests/sessions/replay_consistency/session_memory_summary_diff_report.jsonTest Coverage | 测试覆盖
Self-test Checklist | 自测清单
python -m json.tool tests/sessions/replay_consistency/session_memory_summary_diff_report.jsonpython -m pytest tests/sessions/test_replay_consistency.py tests/sessions/test_replay_mutations.py -q$env:TRPC_AGENT_REPLAY_SQL_URL='sqlite:///:memory:'; python -m pytest tests/sessions/test_replay_consistency.py -q$env:TRPC_AGENT_REPLAY_REDIS_URL='redis://localhost:6379/15'; python -m pytest tests/sessions/test_replay_consistency.py -q