feat: improve RAG retrieval and adopt Helm upgrades

[kmr-rohit]

Summary

• Stabilize the single-GPU Qwen deployment with Standard mode, a persistent Hugging Face cache, explicit ephemeral-storage limits, and a stable service.
• Improve MCP retrieval with bounded queries, hybrid reranking, trusted-source filtering, coherent context expansion, and prompt-injection evidence boundaries.
• Preserve the structured citation contract from PR feat(chatbot): surface MCP citations in the docs-site UI #237, buffer fragmented SSE events in the widget, and require golden structured citations in the showcase evaluator.
• Add separately versioned docs-agent and qwen-runtime Helm releases and migrate OKE CD to Helm upgrade/install.
• Serialize production deployment jobs, reject reused SemVer image tags, retain the MCP canary through functional smoke tests, and add best-effort Qwen migration recovery.

Deployment behavior

Normal merges upgrade only the docs-agent release and do not restart Qwen. The qwen-runtime release runs only from workflow_dispatch with deploy_kserve enabled because the single GPU requires an intentional downtime window.

The KServe Hugging Face runtime remains intentionally pinned at v0.15.2-gpu, matching the currently validated cluster runtime. A runtime image upgrade should be reviewed and tested separately.

No ingestion pipeline is submitted by this change.

Rebase

Rebased onto upstream main at 9a1e5ab, including the structured citation UI changes from PR #237. Earlier commits already merged through PR #234 were dropped rather than replayed.

Validation

• 173 pytest tests passed
• Ruff lint and format checks passed
• Python compilation passed
• JavaScript syntax and fragmented-SSE regression tests passed
• docs-agent, qwen-runtime, and gateway-guardrails charts linted and rendered successfully
• Workflow YAML and embedded shell syntax checks passed

Signed-off commits satisfy DCO.

[tarekabouzeid]

That's quite low specifically for questions about installation or steps for debugging.

We can keep it but need to get feedback from community.

[chasecadet]

I think the idea is to provide links to docs and sources that a client-side coding agent or LLM can use. The goal of Flo is to "teach a man (or agent) to fish. Get you to the right docs, answer succinct questions with references. Today we do not have a ton of GPU capacity and are working to err on the side of surgical, direct answers with sources for "learn more" vs. being a free, hosted chatbot oracle. We can expand tokens and context. 1,024 tokens translates to ~750 words or ~120 lines of formatted code/markdown. The hypothesis is that coding agents want structured, dense facts: "Use API X with flags Y. Read more at URL." and long answers fill up the client-side agent’s context window, increasing their costs and making them "forget" earlier instructions. We should watch out for it: if Flo tries to generate a medium-sized code snippet and a detailed breakdown, she might hit the 1,024-word wall and cut off abruptly.

If a query truly requires a massive code walkthrough, Flo should explicitly output:

"Here is the brief setup: [code]. For full implementation details, refer to [URL]."

@kmr-rohit @SanthoshToorpu thoughts? How are we tackling abrupt clipping of context/response?

[tarekabouzeid]

Thanks @kmr-rohit .

/lgtm
/approve

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tarekabouzeid

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [tarekabouzeid]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment