Repository navigation
chore(models): sync Command Code catalog - #166
github-actions[bot] wants to merge 1 commit into
Conversation
karaaslanz
left a comment
There was a problem hiding this comment.
Independent source-to-official-package review at HEAD 47ba19cb629e2b47cdfe2e01deab10e44b930f59.
I downloaded the published command-code@1.82.5 npm tarball with npm pack --ignore-scripts (no package scripts or downloaded CLI code executed), verified its SHA-512 integrity, read its model reference and dist/cli.mjs as text, and independently rederived the four metadata maps used by the catalog sync tool. Compared against the pinned GitHub src/commandcode-catalog.ts blob d92effa0e33cf06e4dc83abfe823a9992c514e5d.
PASS: 92 model IDs, zero mismatches: image input 70/70, reasoning 79/79, reasoning efforts and value arrays 64/64, max output tokens 10/10. The PR changes only README.md and the generated catalog; pricing files were not touched. Reproducible probe SHA-256: 22abb7572208f28138782082980310c6d348e1d229bd7b1b90e5b74c79f71b4a (pi-166-official-npm-metadata-consistency-probe.cjs). Scope is published CLI metadata consistency, not live endpoint, pricing, or full runtime tests.
CI gate: the PR workflows currently report action_required rather than success. I am recording this as a focused COMMENTED verification, not a full PR approval or merge recommendation until CI is resolved. No upstream source changes or lifecycle script execution.
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Memory benchmarkCompared base
Interpretation: Differences larger than their measured MAD: Stable RSS -1.2 MiB ± 0.5 MiB, Physical peak -1.5 MiB ± 0.4 MiB. Extension overhead above pi baseline
Values are medians of 6 alternating, paired runs. The value after Environment: pi
|
pierreraby
left a comment
There was a problem hiding this comment.
Review — head 47ba19cb (automated catalog sync, 2 files)
Verified independently at this exact head.
Live re-derivation with the repository's own tool. npm run check:commandcode-catalog at this head inspects the published command-code@1.82.5 package and compares it against the committed snapshot: no drift on any map — CLI version current, image-capable models 70/70, reasoning 79/79, selectable efforts 64/64, model-specific output limits 10/10, and None for every New/Removed/Changed row.
Local suites. npm ci --ignore-scripts, npm run typecheck, npm run format:check and git diff --check clean; full npm test green, including the real-host suites ([pi-local] PASS on pi 1.3.0, [omp-compat] PASS).
CI on this head is green after the pending runs were approved: 14 success, 1 skipped (CodeQL, Semgrep OSS + pi extension audit, Gitleaks, npm audit, dependency review, lifecycle-script check, typecheck, format, catalog sync, memory benchmark, OMP compatibility), mergeable_state: clean.
Scope of the change — only README.md and the generated src/commandcode-catalog.ts; pricing is intentionally untouched:
- CLI version
1.82.4→1.82.5 - new image modality:
meta/muse-spark-open,nvidia/nemotron-3.5-super-vl - new reasoning models: the two above plus
stealth/cat-jam:free; removed:inclusionai/ling-3.0-flash-sante:free - new reasoning efforts:
low/medium/highfor the two models,low/medium/high/xhigh/maxforstealth/cat-jam:free - output limits:
stealth/cat-jam:free131,072 added,inclusionai/ling-3.0-flash-sante:free32,768 removed
I also read @karaaslanz's independent source-to-official-package verification on this same head (npm tarball fetched with --ignore-scripts, SHA-512 verified, metadata rederived from the package as text: 92 model IDs, zero mismatches, reproducible probe SHA-256 22abb7572208f28138782082980310c6d348e1d229bd7b1b90e5b74c79f71b4a) — it agrees with the live re-derivation above, so the snapshot is confirmed by two independent methods.
Approving. Noted for the record, none of these block:
- No
CHANGELOG.mdentry. CONTRIBUTING reserves the changelog for user-facing behavior; a dropped reasoning flag for a free model and a new 128k-output free model (stealth/cat-jam:free) are arguably user-visible, so a line either here or in the release PR would be welcome. - The verification scope is published-CLI metadata consistency, not live model availability or endpoint behavior —
stealth/cat-jam:freestill has to be reachable for the accounts allowed to use it. - The automation branch is force-pushed by the daily job, so any future head invalidates this review and needs a fresh check.
Independent live re-derivation plus the full local suite at the exact head; struck at 47ba19cb629e2b47cdfe2e01deab10e44b930f59.
|
Thanks for the independent source-to-official-package check, @karaaslanz. The three runs that were sitting in Since your review was scoped to CI being unresolved, would you like to convert it into an approval if you are satisfied with the head? No merge is pending from my side — that stays with patlux. |
karaaslanz
left a comment
There was a problem hiding this comment.
Follow-up to my source-to-official-npm-package review on unchanged head 47ba19cb629e2b47cdfe2e01deab10e44b930f59.
I have now rechecked the live GitHub PR status after the previously blocked Actions jobs were approved. HEAD is unchanged, mergeable=MERGEABLE, mergeStateStatus=CLEAN, and current checks show 14 SUCCESS / 1 expected SKIPPED, including typecheck, formatting, dependency review, lifecycle-script checks, CodeQL, Semgrep, secrets scan, memory benchmark, OMP compatibility and catalog sync reporting.
My own previously recorded reproducible verification remains valid at this exact head: published command-code@1.82.5 tarball SHA-512 verified, 92 model IDs, zero divergence in the generated image (70/70), reasoning (79/79), effort values (64/64) and max-output (10/10) metadata; no pricing changes. @pierreraby additionally reports an independently successful npm test run, including Pi 1.3.0/OMP real-host suites, and independently ran the repository's check:commandcode-catalog without drift. Their full-suite run is their evidence; my source-to-package test is my own.
Given these new results, the limited earlier CI caveat is resolved and I approve the current head as a two-file, machine-generated model catalog snapshot update. This is not a claim to have live-tested availability/pricing for every model. Merge and release remain with @patlux, and a future automation force-push should trigger fresh validation.
|
Closing as superseded by #167, now merged ( #167 carries exactly this PR's catalog metadata (CLI 1.82.5 → 1.83.0 entry set is identical — only the pinned version string differed) plus the reviewed pricing snapshot and its tests, so we merged that one instead of landing both. Credit where it's due: the 1.82.5 metadata delta was independently verified by @karaaslanz (tarball SHA-512, 92 model ids, zero mismatch) and approved in this PR — that review stands as the validation of the catalog content that shipped via #167. Thanks Ömer! |
Automated synchronization with the latest published
command-codeCLI package.This updates only machine-readable compatibility metadata:
x-command-code-versionheaderPricing remains review-only because CLI documentation does not represent every pricing tier and temporary promotion used by the provider.