Python: fix(redis): honour a max_messages retention limit of zero - #7470
Python: fix(redis): honour a max_messages retention limit of zero#7470chinmayv095 wants to merge 2 commits into
Conversation
RedisHistoryProvider documents None as the sentinel for unlimited storage, so max_messages=0 must retain nothing. It retained everything: trimming to -max_messages emits LTRIM key 0 -1, which is Redis's "keep the whole list", and the count > max_messages guard is true for any non-empty list, so the trim ran on every save and did nothing. Negative values were worse than a no-op. max_messages=-5 emitted LTRIM key 5 -1, deleting the five oldest messages on every save while the list still grew without bound. Handle a limit of zero by deleting the key, which is what clear() in this class already does, and reject negative values in __init__ alongside the three ValueErrors it already raises for invalid configuration. None and positive limits are unchanged.
There was a problem hiding this comment.
Pull request overview
Fixes RedisHistoryProvider retention semantics so max_messages=0 does not behave like “unlimited”, and adds validation to reject negative max_messages values.
Changes:
- Update
RedisHistoryProvider.__init__documentation and configuration validation formax_messages(explicitly document0, reject negatives). - Adjust retention enforcement logic in
save_messagesfor themax_messages=0case. - Add unit tests covering negative
max_messagesandmax_messages=0behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| python/packages/redis/agent_framework_redis/_history_provider.py | Adds max_messages validation and implements the special-case retention handling for 0, plus docstring update. |
| python/packages/redis/tests/test_providers.py | Adds tests for negative max_messages and for max_messages=0 retention behavior. |
| if self.max_messages is not None: | ||
| current_count = await self._redis_client.llen(key) # type: ignore[misc] | ||
| if current_count > self.max_messages: | ||
| await self._redis_client.ltrim(key, -self.max_messages, -1) # type: ignore[misc] | ||
| if self.max_messages == 0: | ||
| # LTRIM key 0 -1 keeps the whole list, so a limit of zero cannot be | ||
| # expressed as a trim to -max_messages. | ||
| await self._redis_client.delete(key) # type: ignore[misc] |
| if max_messages is not None and max_messages < 0: | ||
| raise ValueError("max_messages must be None (unlimited) or a non-negative integer") |
| mock_redis_client.llen = AsyncMock(return_value=15) | ||
|
|
||
| with patch("agent_framework_redis._history_provider.redis.from_url") as mock_from_url: | ||
| mock_from_url.return_value = mock_redis_client | ||
| provider = RedisHistoryProvider("mem", redis_url="redis://localhost:6379", max_messages=0) | ||
|
|
||
| await provider.save_messages("s1", [Message(role="user", contents=["msg"])]) | ||
|
|
||
| mock_redis_client.delete.assert_called_once_with("chat_messages:s1") | ||
| mock_redis_client.ltrim.assert_not_called() |
Addresses the automated review on microsoft#7470. With max_messages=0 the previous change still RPUSHed every message and deleted the key afterwards, so the payload reached Redis - and any AOF or replica stream - before being removed, and was briefly visible to other readers. Short-circuit instead: drop any existing history and return before serializing, so nothing is written at all. Also documents the new ValueError in the Raises: section, and asserts in the test that the pipeline is never used.
|
@microsoft-github-policy-service agree |
| # the whole list - and writing first would put the payload in Redis, and in any | ||
| # AOF or replica stream, before deleting it. Drop any existing history and | ||
| # never write the messages at all. | ||
| await self._redis_client.delete(key) # type: ignore[misc] |
There was a problem hiding this comment.
Could this zero-retention path avoid deleting history owned by another provider? _redis_key() omits source_id, so two RedisHistoryProviders with the default prefix share {key_prefix}:{session_id}; because providers persist in reverse order, a zero-limit provider placed before a positive-limit provider deletes the latter's newly written history. Could the delete be scoped to this provider, or deferred until keys include source_id, so enabling zero retention cannot cause cross-provider data loss?
Motivation & Context
RedisHistoryProvider.__init__documentsmax_messagesas "Maximum number of messages toretain per session ... None means unlimited storage". Because
Noneis the documentedsentinel for unlimited,
0has to mean "retain nothing". Today it means "retain everything".save_messagesenforces the limit by trimming to-max_messages:For
max_messages=0that isLTRIM key 0 -1, which is Redis's "keep the whole list". The guardcurrent_count > self.max_messagesis true for any non-empty list, so the trim runs on everysave and does nothing. The one setting that asks for no retention is the only non-
Nonesettingthat never bounds the list, and it does so silently — no error, no warning, just unbounded growth
of
chat_messages:<session_id>.Negative values are worse than a no-op:
max_messages=-5issuesLTRIM key 5 -1, which deletesthe five oldest messages on every save while the list still grows without bound. That destroys
history and still does not enforce a limit.
Description & Review Guide
What are the major changes?
Two, both in
_history_provider.py:save_messagesshort-circuits when the limit is zero: it drops any existing history withdelete(key)— the callclear()in this same class already uses — and returns beforeserializing anything.
LTRIMcannot express "keep nothing", and writing first would put thepayload into Redis, and into any AOF or replica stream, before deleting it. The positive-limit
trim path is untouched by the diff.
__init__rejects a negativemax_messageswith aValueError, alongside the threeValueErrors it already raises for invalid configuration.The docstring now states the
0case explicitly.What is the impact of these changes?
max_messages=Noneandmax_messages=NforN > 0are byte-for-byte unchanged — the existingltrim(key, -N, -1)path is untouched, and the existing tests covering it pass unmodified. Only0and negatives change, and both are currently broken. No key format, serialization or wirechange, so existing deployments are unaffected.
What do you want reviewers to focus on?
Whether rejecting negatives at construction is preferred over clamping them to
0. I chose toraise because the current behaviour deletes data, so silently reinterpreting the value seemed
worse than refusing it, and because
__init__already validates its other arguments this way.Happy to switch to a clamp if you'd rather not add a new raise.
Deliberately out of scope —
RedisHistoryProviderstill writes to{key_prefix}:{session_id}withoutsource_id, so two providers with differentsource_idsbut the same prefix share one list, where
CosmosHistoryProviderisolates bysource_idinget_messages,clearandlist_sessions. That is a key-format change with migrationimplications, not a bug fix, so it is not in this PR. Raised separately as #7471, with the
migration options laid out there.
Related Issue
Fixes #7469
Contribution Checklist
Verification, all on
python/packages/redis:main: 45 passed, 0 failed.test_max_messages_zero_retains_nothing(deletecalled 0 times) andtest_negative_max_messages_raises(noValueError). With the fix: 47 passed, 0 failed,zero regressions.
ruff checkclean,ruff format --checkreports both files already formatted.mypy --config-file python/pyproject.toml agent_framework_redis— no issues in 4 source files.the new
ValueErroris documented underRaises:, and the test now asserts the pipeline isnever used. All checks re-run green.