Skip to content

feat(vchordrq): add pluggable exact MaxSim reranking - #478

Draft
HuXinjing wants to merge 1 commit into
supervc-stack:mainfrom
HuXinjing:upstream/maxsim-exact-rerank
Draft

feat(vchordrq): add pluggable exact MaxSim reranking#478
HuXinjing wants to merge 1 commit into
supervc-stack:mainfrom
HuXinjing:upstream/maxsim-exact-rerank

Conversation

@HuXinjing

Copy link
Copy Markdown

Summary

This is the second implementation slice proposed in Discussion #476.

It introduces a storage-independent exact MaxSim reranking boundary and a CPU reference backend. The reference tensor source reads the indexed expression from the PostgreSQL heap under the active scan snapshot; it does not introduce an external registry, network transport, GPU dependency, or quantized format.

User-visible behavior

  • Existing behavior remains the default through vchordrq.maxsim_backend = 'coarse_only'.
  • Exact reranking is opt-in with vchordrq.maxsim_backend = 'cpu_exact'.
  • Exact mode requires an explicit positive vchordrq.maxsim_candidate_limit, bounded at 65,536.
  • Candidate generation remains the existing vchordrq MaxSim path. Only the bounded candidate set is rescored using the full heap tensor.

Architecture boundary

  • CandidateTensorSource separates full-tensor retrieval from scoring.
  • ExactMaxsimBackend separates exact scoring from candidate generation and storage.
  • HeapTensorSource is the only source in this PR.
  • CpuExactMaxsimBackend is the only backend in this PR.

External tensor-source registration and any optional GPU process remain separate follow-up proposals, contingent on maintainer feedback.

Correctness and safety

  • Uses the existing HeapFetcher, active PostgreSQL snapshot, and indexed-expression evaluation.
  • Rechecks the active scan qual before materializing a candidate tensor.
  • Rejects null, mismatched-kind, and mismatched-dimension tensors.
  • Does not add background workers, sockets, persistent state, migrations, or build dependencies.

Validation

  • cargo fmt --all -- --check
  • cargo test --locked --workspace --exclude vchord --no-fail-fast
  • PG16 release extension build and generated SQL schema
  • PG16 root-extension unit test for exact result ordering
  • Added tests/vchordrq/maxsim_cpu_exact.slt

Local validation gap: the isolated PG16 build image contains development headers but no initdb, so the new SQLLogicTest was not executed against a locally started PostgreSQL server. It is expected to run in the repository CI PostgreSQL matrix.

Relationship to other work

This branch is independently based on current main; it does not depend on #477.

@github-actions

github-actions Bot commented Aug 23, 2026

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@HuXinjing

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

gaocegege added a commit to tensorchord/CLA that referenced this pull request Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant