Skip to content

test(journey): tolerate server default RQ quantizer - #469

Merged
bevzzz merged 1 commit into
weaviate:mainfrom
dudanogueira:fix/journey-default-quantizer
Sep 3, 2026
Merged

bevzzz merged 1 commit into
weaviate:mainfrom
dudanogueira:fix/journey-default-quantizer

Conversation

@dudanogueira

Copy link
Copy Markdown
Contributor

Motivation

test/collections/journey.test.ts connects to a fixed WCD cluster (piblpmmdsiknacjnm1ltla.c1.europe-west3.gcp.weaviate.cloud), not the local container spun up for CI. That cluster was upgraded to a server version that applies a default RQ quantizer to HNSW indexes, so the strict toEqual<CollectionConfig> on the returned config started failing with quantizer: undefined (expected) vs { type: 'rq', bits: 8, rescoreLimit: 20 } (actual).

Because the cluster is external and shared, this failed every tests-without-auth matrix job simultaneously — all seven Node/Weaviate combinations, on every open PR — regardless of the Weaviate version under test. The red CI is not caused by anything in this repo, and it blocks all other work (e.g. #459, #466).

Approach

Pull quantizer out of the deep-equal, assert its shape loosely (expect(['rq', undefined]).toContain(quantizer?.type)), and feed the actual value back into the deep-equal so everything else in the config is still compared strictly. This keeps the test meaningful — it still catches a config field going wrong — while making it agnostic to which side of the default-quantizer change the cluster happens to be on.

This mirrors the approach already taken in fa9c488 for the openai hfresh expectation in integration.test.ts, so the codebase now handles the same server-side variability consistently in both places.

Alternatives rejected:

  • Pinning the expectation to RQ: would break again the moment the journey test runs against an older server, and would break contributors running the suite locally.
  • Configuring an explicit quantizer in the test's collections.create: changes what the test covers — the point of this assertion is the default config the server returns, not one the client dictated.

Key areas for review

  • test/collections/journey.test.ts:59-63 — the destructure + loose assertion. Worth confirming the VectorIndexConfigHNSW cast is right here (the collection is created with default HNSW, not hfresh) and that quantizer?.type is the narrowest thing we can usefully assert.
  • test/collections/journey.test.ts:210 — quantizer is now threaded into the deep-equal; this is what preserves strictness for the remaining fields.

Risks and mitigations

Low risk — test-only, one file, no src/ changes.

  • Weakened coverage: the only thing no longer strictly asserted is the quantizer's bits/rescoreLimit; its presence and type are still checked, and integration.test.ts covers quantizer configuration against the local container where the server version is known.
  • Masking a real regression: undefined is still an accepted value, so a client-side bug that drops the quantizer entirely would pass here. Accepted deliberately — the journey test targets an external cluster we don't version-control, so it can't be the place that pins quantizer behaviour.

Testing

Verified against the live WCD cluster the test targets: should get the config for the created collection now passes where it previously failed on the quantizer mismatch, with the rest of the config deep-equal unchanged. No changes to src/, so no other suites are affected.

Follow-up (not in this PR)

The cluster URL and API key are hardcoded in plaintext at test/collections/journey.test.ts:19-23 and have been in this public repo since 2024-05-23. Worth a separate issue to move them to Actions secrets — and to reconsider whether a shared, externally-mutable cluster belongs in the blocking CI path at all, given this failure mode (an external upgrade reddens every PR at once) will recur.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SXVS2uKtuEH91QEVka8zWV

@orca-security-eu orca-security-eu Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Passed SAST high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca

@OsamaAnsar

Copy link
Copy Markdown
Contributor

it also unblocks #464 and #468

Comment thread test/collections/journey.test.ts

@bevzzz bevzzz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dudanogueira please please please stop writing these massive PR descriptions. It's a 4 line change in the test code. There's no need to describe Approach, Risks, Testing, etc; all it does is waste the reviews' time.

The WCD cluster this journey test targets now applies a default RQ
quantizer to HNSW indexes, so the strict deep-equal on the collection
config fails with `quantizer: undefined` vs `{type: 'rq', bits: 8,
rescoreLimit: 20}`.

Because the test connects to a fixed cloud cluster rather than the local
container, this broke every `tests-without-auth` matrix job at once, on
every Weaviate version, across all open PRs.

Pull `quantizer` out of the deep-equal and assert its shape loosely,
matching the approach already taken in fa9c488 for the openai hfresh
expectation. The rest of the config is still compared strictly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SXVS2uKtuEH91QEVka8zWV
@dudanogueira
dudanogueira force-pushed the fix/journey-default-quantizer branch from 6f9a442 to ac72d67 Compare September 2, 2026 21:36
@bevzzz
bevzzz merged commit 20e1846 into weaviate:main Sep 3, 2026
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants