Skip to content

feat(rag): hybrid docs retrieval and readable ingest path - #249

Open
SanthoshToorpu wants to merge 1 commit into
kubeflow:mainfrom
SanthoshToorpu:feature/rag-performance
Open

feat(rag): hybrid docs retrieval and readable ingest path#249
SanthoshToorpu wants to merge 1 commit into
kubeflow:mainfrom
SanthoshToorpu:feature/rag-performance

Conversation

@SanthoshToorpu

Copy link
Copy Markdown
Contributor

Summary

  • Index Kubeflow docs as hybrid Milvus rows (dense + BM25 + release_date) and route queries in MCP instead of dense-only search.
  • Keep ingest as importable modules in a slim image; kubeflow-pipeline.py is the DAG and milvus_store.py writes the collection (v=4 is only the schema stamp).
  • Split MCP into router / search / citations, park incremental ingest under legacy/, and record the eval case in docs/RAG_V4_ARCHITECTURE.md.

Test plan

  • pytest tests/test_mcp_server.py tests/test_kubeflow_pipeline_v4.py tests/test_canonical_rag_ingest.py tests/test_hugo_ingest.py
  • python docs-agent-mcp/pipelines/kubeflow-pipeline.py compiles
  • Chatbot Sources panel still renders citations after MCP deploy
  • Do not run incremental pipeline against production kubeflow_docs
  • Cluster ingest run needs the docs-rag-ingest image built separately (not part of MCP CD)

Made with Cursor

Dense-only search misses versions and release dates. Write hybrid
Milvus rows, route queries in MCP, and keep ingest as importable
modules in a slim image instead of pasted KFP source.

Co-authored-by: Cursor <cursoragent@cursor.com>
@google-oss-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign chasecadet for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant