Skip to content

fix(local): remove keys set to null in set_payload, like the server - #1530

Open
Yuilona wants to merge 1 commit into
qdrant:devfrom
Yuilona:fix/local-set-payload-null
Open

Yuilona wants to merge 1 commit into
qdrant:devfrom
Yuilona:fix/local-set-payload-null

Conversation

@Yuilona

@Yuilona Yuilona commented Oct 4, 2026 •

Copy link
Copy Markdown

When set_payload receives a top-level null value, the server removes that key from the target object; local mode stores the null instead.

from qdrant_client import QdrantClient, models

for client in (QdrantClient("http://localhost:6333"), QdrantClient(":memory:")):
    if client.collection_exists("demo"):
        client.delete_collection("demo")
    client.create_collection("demo", vectors_config=models.VectorParams(size=2, distance=models.Distance.DOT))
    client.upsert("demo", [models.PointStruct(id=1, vector=[1, 0], payload={"a": 1, "keep": 1})], wait=True)
    client.set_payload("demo", payload={"a": None, "b": None}, points=[1], wait=True)
    print(client.retrieve("demo", [1])[0].payload)
server: {'keep': 1}
local:  {'a': None, 'keep': 1, 'b': None}

On the server, every set_payload ends in merge_map, which removes a key whose value is Value::Null instead of inserting it. That covers the no-key case (Payload::merge) and every path handled by JsonPath::value_set, including paths whose target is not an object and gets replaced by a new one. Local mode merged with dict.update() / {**a, **b} and kept the nulls. The same applies to SetPayloadOperation in batch_update_points.

This adds a merge_payload helper with the server's semantics and uses it everywhere local mode merges a set_payload value: the no-key path and the terminal writes of the key, index and wildcard setters. Nulls nested inside a value ({"e": {"f": None}}) are still stored, as on the server. overwrite_payload and upsert keep nulls on both sides and are not changed.

Tests

  • test_set_value_by_key_null_removes_key: one case per setter branch (dict target, non-dict target replaced, missing path, index, wildcard). All 7 fail before this change.
  • test_set_payload_null_removes_key (congruence): the same request with no key and with b, a.x, arr[0], arr[1], arr[] and new.path, over REST and gRPC. It fails before this change and passes after.

Local runs against Qdrant v1.19.1 (merge_map and value_set are identical on dev):

pytest qdrant_client/local/tests                      146 passed
pytest tests/congruence_tests/test_payload.py tests/congruence_tests/test_updates.py \
       tests/congruence_tests/test_delete_points.py tests/congruence_tests/test_uuids.py   68 passed

I found this with a small script that runs random set_payload / overwrite_payload / delete_payload / clear_payload calls against both a server and :memory: and compares the payloads afterwards. Before the change, 7 of 300 rounds diverged, all because of nulls; after the change, 0 of 900 diverged. ruff format (v0.4.3, line length 99) and mypy report no issues for the changed files.

All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

AI-assisted: the comparison script, fix and tests were prepared and run with Claude Code.

🤖 Generated with Claude Code

The server merges a set_payload request into the target object with
merge_map, which removes a key whose new value is null instead of storing
the null. Local mode used dict.update() and stored the nulls, so the same
request left different payloads on the server and in local mode. This holds
with and without `key`, and for the paths that replace a non-object target.

Nulls nested inside a value are still stored as is, as on the server.
overwrite_payload and upsert keep nulls on both sides and are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Yuilona
Yuilona requested a review from joein as a code owner October 4, 2026 20:32
@netlify

netlify Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for poetic-froyo-8baba7 ready!

Name Link
🔨 Latest commit 5f6e3f2
🔍 Latest deploy log https://app.netlify.com/projects/poetic-froyo-8baba7/deploys/6ac2b7e746c54600082d5879
😎 Deploy Preview https://deploy-preview-1530--poetic-froyo-8baba7.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: eb73eeeb-2d84-4c1c-b370-0d54ff9f4fdf
📥 Commits

Reviewing files that changed from the base of the PR and between d64b76d and 5f6e3f2.

📒 Files selected for processing (4)
  • qdrant_client/local/local_collection.py
  • qdrant_client/local/payload_value_setter.py
  • qdrant_client/local/tests/test_payload_utils.py
  • tests/congruence_tests/test_payload.py

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.


📝 Walkthrough

Walkthrough

Payload updates now remove destination keys when the matching source value is null. This behavior applies to unkeyed local collection updates and terminal key, index, and wildcard paths. Non-dictionary values at terminal paths are replaced with dictionaries that omit null-valued source keys. Tests cover local path cases and compare local and remote collection results.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~12 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 5f6e3

The null-removal change has no identified issue that needs resolution before merge. Proceed with normal checks.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 5f6e3

The change intentionally replaces stored nulls with key deletion, so callers relying on the previous local behavior will observe different payloads. The inspected implementation preserves existing point selection and write scope; no new security boundary or privilege expansion was identified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The changed operation affects payloads of the selected existing points. A wildcard can update every element of its resolved array, and incompatible targets can be replaced, but those scopes predate this PR. The inspected change does not extend selection to other collections, services, or credentials.

Trust Boundaries and Controls

  • inferred — Caller-supplied payloads and paths reach the existing selected-point mutation path. Null now means deletion inside that write scope, but does not grant a new operation over otherwise unreachable points or bypass the missing/deleted-point check. Authorization outside this local path was not established by the supplied topology.

Resilience and Maintainability Implications

  • inferred — Mutation precedes persistence and commits remain per point, so interruption or persistence failure can leave a partially completed request. This ordering predates the null-removal change. Static inspection does not establish serialized concurrent calls or crash-tested recovery guarantees, and neither guarantee is added here.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 4 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: local set_payload now removes keys set to null, matching server behavior.
Description check ✅ Passed The description explains the behavior difference, the fix, and the tests. It is directly related to the changeset.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant