build(deps): update crawlkit to v0.16.6

[steipete]

Update CrawlKit from v0.16.5 to the published v0.16.6 tag for faster vector validation in the TurboVec search backend. Refresh the source-build version references and add the Unreleased note; gitcrawl's existing prepared/SIMD exact scoring is unchanged.

The only module version change is github.com/openclaw/crawlkit v0.16.5 → v0.16.6, with its matching checksums. SQLite remains v1.59.0, libc remains v1.75.7, and every other direct/indirect version is unchanged.

Validation: build, tidy-check, formatting, vet, vulnerability/deadcode checks, smoke, release-script tests, documentation tests/build, and six-platform snapshot builds passed. Full tests and coverage passed on Linux (85.8%), along with the SIMD full suite and forced scalar fallback tests. One portable-Git context timeout on a heavily loaded Mac passed on the CLI-package retry. Codex review: scoped-clean at P0/P1.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed September 29, 2026, 9:46 PM ET / September 30, 2026, 01:46 UTC.

ClawSweeper review

What this changes

Updates the CrawlKit Go module to v0.16.6, replaces its checksums, and refreshes source-build documentation and the unreleased note.

Merge readiness

✅ Ready for maintainer review

Current main still uses CrawlKit v0.16.5, and no separate item owns this update. The PR is a coherent, bounded dependency upgrade with no concrete introduced defect found.

Priority: P3
Reviewed head: 1fa6620fe60bb3a121aaff19f602942e903e0600

Review scores

Measure	Result	What it means
Overall readiness	🐚 platinum hermit (4/6)	The small, coherent update has a published upstream source delta and no actionable finding; external-contributor proof is not required.
Proof confidence	🌊 off-meta tidepool	Not applicable: Gitcrawl's vector search is the relevant production path, and the PR body reports validation of the updated dependency. The repository administrator's PR is exempt from external-contributor proof; upstream byte-equivalence coverage supports that the encoding implementation does not change the stored-data contract.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Verification

Check	Result	Evidence
Real behavior	Not applicable	Not applicable: Gitcrawl's vector search is the relevant production path, and the PR body reports validation of the updated dependency. The repository administrator's PR is exempt from external-contributor proof; upstream byte-equivalence coverage supports that the encoding implementation does not change the stored-data contract.
Evidence reviewed	9 items	Current main: The pinned main revision requires CrawlKit v0.16.5, so the requested update is not already present. Introduced module update: The pinned PR delta changes the direct CrawlKit requirement to v0.16.6 and updates its two checksum entries; no other module version changes. Dependency boundary: Gitcrawl scores exact results locally and calls CrawlKit search when the optional TurboVec backend is selected.
Findings	None	None.
Security	None	None.

How this fits together

Gitcrawl searches stored GitHub archive embeddings through a local exact scorer or an optional TurboVec backend. CrawlKit supplies shared vector APIs used by that search path and other Gitcrawl components.

flowchart LR
  A[Search request] --> C[Backend choice]
  B[Stored embeddings] --> C
  C -->|Exact| D[Local vector scoring]
  C -->|TurboVec| E[CrawlKit search]
  D --> F[Ranked results]
  E --> F

Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Dependency scope	1 direct module updated; 0 other module versions changed	The version change is confined to the published CrawlKit patch release.

Technical review

Best possible solution:

Ship the published CrawlKit release while preserving Gitcrawl's existing backend choices and vector-search behavior.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this is a dependency update rather than a reported failure; the old and new module pins are directly verifiable.

Is this the best way to solve the issue?

Yes: updating the published module pin and matching checksums is the narrow path, and the PR leaves Gitcrawl's backend selection intact.

AGENTS.md: found, but no applicable review policy affected this item.

Codex review notes: model internal, reasoning high; reviewed against 6ccd9334ccb0.

Labels

Label changes:

• add P3: This is a bounded dependency and documentation update with no demonstrated urgent user regression.
• add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
• add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: Gitcrawl's vector search is the relevant production path, and the PR body reports validation of the updated dependency. The repository administrator's PR is exempt from external-contributor proof; upstream byte-equivalence coverage supports that the encoding implementation does not change the stored-data contract.

Label justifications:

• P3: This is a bounded dependency and documentation update with no demonstrated urgent user regression.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: Gitcrawl's vector search is the relevant production path, and the PR body reports validation of the updated dependency. The repository administrator's PR is exempt from external-contributor proof; upstream byte-equivalence coverage supports that the encoding implementation does not change the stored-data contract.

Evidence

What I checked:

• Current main: The pinned main revision requires CrawlKit v0.16.5, so the requested update is not already present. (go.mod:13, 6ccd9334ccb0)
• Introduced module update: The pinned PR delta changes the direct CrawlKit requirement to v0.16.6 and updates its two checksum entries; no other module version changes. (go.mod:13, 1fa6620fe60b)
• Dependency boundary: Gitcrawl scores exact results locally and calls CrawlKit search when the optional TurboVec backend is selected. (internal/vector/exact.go:125, 6ccd9334ccb0)
• Upstream behavior: The v0.16.5-to-v0.16.6 comparison changes vector validation, arithmetic, and encoding implementations while retaining the search API used by Gitcrawl. (vector/search.go:80, 610c54c15ab1)
• Stored-format check: The upstream encoding test compares the new float32 byte output and decoded bits with the previous binary implementation, including edge values. No stored-format change is established. (vector/encoding_test.go:12, 610c54c15ab1)
• Published dependency: CrawlKit v0.16.6 is a published release; its annotated tag resolves to the upstream commit cited above. (610c54c15ab1)

Likely related people:

• steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
• vincentkoc: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.