fix(cache): prevent over-invalidation of similarly named prompt prefixes in PromptCache - #1829
Open
Xayar145 wants to merge 4 commits into
Open
fix(cache): prevent over-invalidation of similarly named prompt prefixes in PromptCache#1829Xayar145 wants to merge 4 commits into
Xayar145 wants to merge 4 commits into
Conversation
…xes in PromptCache
|
|
| with self._lock: | ||
| for key in list(self._cache): | ||
| if key.startswith(prompt_name): | ||
| if key == prompt_name or key.startswith(prefix): |
Contributor
There was a problem hiding this comment.
Hyphen-delimited names still collide
For cached prompts named summary and summary-detailed, invalidating summary also matches summary-detailed-version:1, unnecessarily evicting the distinct sibling and forcing another API fetch on its next access. The added underscore case does not cover this collision because prompt names accept hyphens and cache metadata uses the same delimiter.
Knowledge Base Used: Utils and Support Types
Prompt To Fix With AI
This is a comment left during a code review.
Path: langfuse/_utils/prompt_cache.py
Line: 186
Comment:
**Hyphen-delimited names still collide**
For cached prompts named `summary` and `summary-detailed`, invalidating `summary` also matches `summary-detailed-version:1`, unnecessarily evicting the distinct sibling and forcing another API fetch on its next access. The added underscore case does not cover this collision because prompt names accept hyphens and cache metadata uses the same delimiter.
**Knowledge Base Used:** [Utils and Support Types](https://app.greptile.com/personal-org-4986/-/custom-context/knowledge-base/langfuse/langfuse-python/-/docs/utils-support.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes cache invalidation in
PromptCache.invalidate():{prompt_name}-instead of bare substring prefix matching (key.startswith(prompt_name)).test_prompt_cache_invalidate_exact_prefix_match.Motivation
Previously,
PromptCache.invalidate("summary")would unintentionally evict other prompt keys sharing the same name prefix (for example"summary_detailed-version:1"or"summary_qa-label:production"). Checking exact key match or hyphen-delimited prefix ensures only the target prompt is evicted.Changes
langfuse/_utils/prompt_cache.py: Refactoredinvalidate()to matchkey == prompt_name or key.startswith(f"{prompt_name}-").tests/unit/test_prompt.py: Added test ensuring sibling prompts with similar prefixes are not accidentally evicted.Tests
test_prompt_cache_invalidate_exact_prefix_matchverifying selective invalidation.Greptile Summary
The PR narrows prompt-cache invalidation to exact keys and hyphen-delimited prefixes and adds a regression test for similarly prefixed names. The delimiter remains ambiguous for valid prompt names containing hyphens.
PromptCache.invalidate()to avoid bare prefix matching.summarypreservessummary_detailed.summary-detailed.Confidence Score: 4/5
The PR is safe to merge with a non-blocking cache-correctness improvement recommended for hyphen-prefixed sibling prompt names.
The change fixes underscore-style prefix collisions, but the shared hyphen delimiter still causes distinct names such as
summary-detailedto be evicted whensummaryis invalidated.Files Needing Attention: langfuse/_utils/prompt_cache.py, tests/unit/test_prompt.py
Prompt To Fix All With AI
Reviews (1): Last reviewed commit: "test(prompt): add unit test for exact pr..." | Re-trigger Greptile
Context used: