Course
llm-zoomcamp
Question
Why do we use (doc["filename"], doc["start"]) as the key when combining search results with RRF in the LLM Zoomcamp hybrid search implementation?
Answer
In the hybrid search implementation, each document can be split into multiple chunks. Two chunks can therefore come from the same filename but have different start positions.
For this reason, the RRF implementation uses:
key = (doc["filename"], doc["start"])
instead of using only doc["filename"].
The combination of filename and start uniquely identifies a chunk within the indexed data. This allows RRF to correctly match the same chunk when it appears in both the text-search and vector-search result lists.
If we used only filename, different chunks from the same document would be treated as the same result and their RRF scores could be incorrectly combined.
So the tuple (filename, start) is used because RRF is ranking and combining chunks, not entire source files.
Checklist
Course
llm-zoomcamp
Question
Why do we use
(doc["filename"], doc["start"])as the key when combining search results with RRF in the LLM Zoomcamp hybrid search implementation?Answer
In the hybrid search implementation, each document can be split into multiple chunks. Two chunks can therefore come from the same
filenamebut have differentstartpositions.For this reason, the RRF implementation uses:
instead of using only
doc["filename"].The combination of
filenameandstartuniquely identifies a chunk within the indexed data. This allows RRF to correctly match the same chunk when it appears in both the text-search and vector-search result lists.If we used only
filename, different chunks from the same document would be treated as the same result and their RRF scores could be incorrectly combined.So the tuple
(filename, start)is used because RRF is ranking and combining chunks, not entire source files.Checklist