Repository navigation
chore(models): refresh CLI 1.87.2 and reviewed display pricing - #169
Conversation
Bump the static capability snapshot to command-code@1.84.0, which publishes no model metadata changes, and add reviewed display pricing for the new free reflection/beam:free model. Beam is free only while capacity lasts, with no published expiry, and it has no CLI capability metadata yet, so it keeps the conservative text-only and non-reasoning defaults. Re-verified display prices against the official pricing page on 2026-10-17; the snapshot now covers all 93 advertised models.
pierreraby
left a comment
There was a problem hiding this comment.
Review: catalog 1.84.0 + reviewed Beam pricing
Verified at the exact head 0cbb3fce18be2be5f736ea2c0460c68304727414 (1 commit over main f993145), 7 files +14/−8, no .github/ change, mergeable_state: clean, CI 14 pass / 2 skipped (Update PR comment, sync). No review existed before this one.
What I reproduced locally (isolated worktree, npm ci --ignore-scripts)
npm run typecheck,npm run format:check,git diff --check: clean.npm test: exit 0, including[pi-local] PASSand[omp-compat] PASSon the real hosts.npm run check:commandcode-catalog: metadata is current against livecommand-code@1.84.0— 70/70 image-capable, 79/79 reasoning, 64/64 with selectable efforts, 10/10 output limits, and every change row isNone.npm run check:commandcode-pricing: PASS, 93 models, no drift,reflection/beam:freematched thebeam-freepromotion (100%, while capacity lasts).- Red on base: the PR's
tests/test-pricing.tsagainstmain'ssrc/and fixtures fails in exactly 2 places (catalog coverage + provenance), everything else stays green, so the new assertions really pin this change.
Independent counter-checks (not relying on the repo's own parser)
- CLI tarballs
1.83.0vs1.84.0:dist/bundled/command-code-knowledge/reference/models.mdis byte-identical, so the "no model metadata change" claim holds; the generated catalog diff is onlyCOMMAND_CODE_CLI_VERSIONplus its doc comment. beamappears nowhere in the1.84.0package, so the conservative text-only, non-reasoning defaults are the only defensible choice, and the PR says exactly that.- Live
/provider/v1/models: 93 models,reflection/beam:free("Beam 1.0",context_length: 1048576,supported_endpoints: ["/chat/completions", "/responses"]). - Official pricing page,
#beam-free: 100% off, "Free while capacity lasts", Input / Output / Cache read all$0.00, no expiry → a plain{0,0,0,0}entry with a comment (not aTEMPORARY_PRICINGdeadline) is the right modelling, including the "no published expiry" caveat. - Because the live pricing check re-verifies all 93 advertised models, the global
PRICING_LAST_VERIFIED → 2026-10-17bump is validated rather than assumed.
Notes (non-blocking)
- P2 — the new rate has no named reviewed-rate assertion. Every catalog cohort in
tests/test-pricing.tssince September has its ownassertCostblock — including the free ones,stealth/glyph-cluster:free(October) andstealth/cat-jam:free(mid-October) — and.agents/skills/refresh-model-catalog/SKILL.mdstep 5 lists "per-model rate assertions" as part of the refresh. Beam only lands in thefreeModelsset. Sincetests/fixtures/commandcode-pricing.jsonis generated fromsrc/pricing.ts, thedeepEqualattests/test-pricing.ts:96is a source↔fixture consistency check rather than a review anchor; the named blocks are what a human actually signed off. A one-lineassertCost("reflection/beam:free", { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 })in a late-October block would close it. The zero rates are still pinned transitively by the free-set loop, so this is a provenance gap, not a correctness hole. - P2 — what a Beam user actually gets today. The live catalog gives the 1M context window (it is read from
context_length), but with no CLI capability metadata the model surfaces as text-only and non-reasoning, and its output limit falls back toDEFAULT_MAX_OUTPUT_TOKENS(65,536). The official page describes Beam 1.0 as a reasoning model withlow–maxefforts, so the gap is user-visible until the daily sync picks the metadata up. The CHANGELOG states the defaults; a one-clause "effort selection appears with the next catalog sync" would make the expectation explicit. - P3 — the fixture date assertion stays brittle.
assert.match(fixture.fetchedAt, /^2026-10-17T/)is a pre-existing pattern that this PR has to bump by hand; it pins a literal day instead of the shape or freshness of the snapshot. The base already shows why deriving it fromPRICING_LAST_VERIFIEDwould be wrong (fetch day 10-16, verify day 10-15), so the cheap improvement is to keepPRICING_LAST_VERIFIEDas the single date pin and relax thefetchedAtassertion to a format check. - P3 — "while capacity lasts" cannot be enforced by the repo's mechanism.
TEMPORARY_PRICINGrequires aYYYY-MM-DDexpiresOn, so a condition-based end cannot be encoded; the new comment plus the empty temporary list means the only trigger is the dailycheck:commandcode-pricingdrift when the page starts charging. That matches the existingcat-jam/glyph-clusterpattern, so it is acceptable, but anendsWhenfield (or a checker note) would make the condition machine-visible if Beam ever stops being free before its metadata lands. - P3 — no unit coverage for
:freepage-id matching. The pricing checker derives the page idbeam:freefromreflection/beam:free, and the matching tests never exercise aprovider/name:freeid; the captured page fixture predates every free model. A live run does match it (beam-free, 100%, while capacity lasts), so this is a testing gap rather than a risk for this PR.
Second opinion
An independent review pass was run on the same head. Its environment had no shell, so I re-checked its substantive claims myself: the four generated catalog maps are byte-identical between 1.83.0 and 1.84.0, and the three artifacts agree at 93 models (commandcode-model-ids.json, MODEL_COSTS, the pricing fixture). It converged on the findings above and added the :free page-id and TEMPORARY_PRICING notes; it found nothing correctness-breaking.
What this PR also restores
The scheduled Command Code catalog sync on main failed on 2026-10-17 with missing-local-price | reflection/beam:free and the snapshot one CLI version behind. This PR is what turns that signal green again, and it is a prerequisite for the check gate of any new PR once CI runs against command-code@1.84.0.
pierreraby
left a comment
There was a problem hiding this comment.
Re-verified at the new head 83ea142c6281008cddc3cfdd582284c6fcf5d6a0 — my earlier approval at 0cbb3fc was auto-dismissed by the push, as expected.
Delta since 0cbb3fc: 3 files, +33/−2, no src/ change.
tests/test-pricing.ts— adds the namedassertCost("reflection/beam:free", { 0, 0, 0, 0 })I asked for, and replaces the literal^2026-10-17Tfixture-date regex with a full ISO-8601 shape check plus atoISOString()round-trip. Both were review notes, now addressed.tests/test-pricing-check.ts— new unit test for the provider-prefixed:freeid:pricingPageIdCandidates("reflection/beam:free")yields[reflection/beam:free, beam:free], and the capacity-limitedbeam-freedeal (100 % off, no expiry) is preserved withexpires === undefined.CHANGELOG.md— now states the 65,536-token output default and that reasoning/effort selection arrives with upstream CLI metadata.
Verification at the exact head (isolated worktree, npm ci --ignore-scripts, clean tree):
npm run typecheckexit 0,npm run format:checkclean,git diff --checkclean.npm testexit 0 — full suite, including[pi-local] PASSand[omp-compat] PASS.npm run check:commandcode-catalogexit 0 against the livecommand-code@1.84.0: 70/70 image, 79/79 reasoning, 64/64 efforts, 10/10 output limits, every change rowNone.npm run check:commandcode-pricingexit 0: 93 models checked,Status: PASS, no drift, andreflection/beam:freematched to thebeam-freedeal (100 % off, "while capacity lasts", no expiry).
Honest note on this delta: it is test-coverage and wording only, and the two touched test files also pass at the parent head 0cbb3fc (11/11 in both cases) — so these assertions are useful anchors, not red/green regression evidence for this change. That is expected here, since the Beam price entry itself landed in the original commit.
One residual note, not blocking: tests/fixtures/commandcode-pricing.json is regenerated from src/pricing.ts (refresh-model-catalog skill), so the deepEqual(MODEL_COSTS, expected) assertion is self-referential and cannot independently anchor those numbers. The named assertCost entries plus the live check:commandcode-pricing run are what actually catch pricing drift — which this PR now does for Beam.
No blocking findings. Merge decision stays with the maintainers; note that main's daily Command Code catalog sync will keep failing on missing-local-price | reflection/beam:free (run 39121673136) until this PR or the bot sync lands.
|
While this PR was waiting to merge, the upstream model catalog changed again. I rechecked the live API and official pricing page on 2026-10-20 and pushed the follow-up in 64f7285. Changes compared with the previous PR revision:
The latest published CLI is still Verification: catalog checker current; pricing checker PASS for 94 models; full |
pierreraby
left a comment
There was a problem hiding this comment.
Re-verified at 64f72852b87491eb1a97d9f635a75170780e91f6. This supersedes my earlier approvals on 0cbb3fc and 83ea142, both auto-dismissed by the later pushes.
Delta since my last approval (83ea142..64f7285: 5 files, +44/−7; the only src/ change is src/pricing.ts):
- paid
minimax/minimax-m3.1-flash— $0.15 input / $0.60 output / $0.015 cache-read, no cache-write; minimax/minimax-m3.1-flash:freeat zero cost plus aTEMPORARY_PRICINGentry;stealth/cat-jam:freedisplay price removed;PRICING_LAST_VERIFIED2026-10-17 → 2026-10-20, fixtures and tests refreshed accordingly.
Local verification (isolated worktree, npm ci --ignore-scripts) at the exact head:
npm run typecheckPASS,npm run format:checkPASS,git diff --checkclean;npm testPASS (exit 0), including the real-Pi integration suite ([pi-local] PASS, pi 1.3.0) and[omp-compat] PASS;npm run check:commandcode-catalog→CLI version: Currentagainstcommand-code@1.84.0: 70/70 image, 79/79 reasoning, 64/64 selectable efforts, 10/10 output limits, every New/Removed/Changed rowNone;npm run check:commandcode-pricing→Status: PASS, 94 models, no drift. The promotions table now listsminimax/minimax-m3.1-flash:free/minimax-m3-1-flash-freeat 100% withexpires 2026-10-21T10:45:00Z, and no longer listscat-jam-free.
Independent cross-checks (not taken from the PR body):
- live
GET /provider/v1/models→ 94 ids, containingminimax/minimax-m3.1-flashandminimax/minimax-m3.1-flash:free. The-previewspelling the 10-19 sync run reported is gone upstream, andstealth/cat-jam:freeis no longer advertised; - official pricing page payload →
minimax-m3.1-flashinputCost 0.15 / outputCost 0.6 / cacheReadCost 0.015with no cache-write field, and the free dealminimax-m3-1-flash-freeisdiscountPercent 100,starts 2026-10-19T11:00:00Z,expires 2026-10-21T10:45:00Z. The page carries nocat-jam-freedeal any more; - RED-before-GREEN on the delta: the parent
83ea142sources plus this head'stests/test-pricing.tsand fixtures maketest:pricingexit 1 with exactly the expected assertions (MODEL_COSTSdeep-equal,minimax/minimax-m3.1-flash should have pricing,stealth/cat-jam:free free-model status should be explicit,PRICING_LAST_VERIFIED), and 12/12 pass at the head.
Notes (non-blocking):
- P2 — the date-only guard trips at
2026-10-21T00:00Z, about 10h45 before the promotion actually ends. Merged as-is,npm testlegitimately goes red onmainin the early hours of 2026-10-21. That is the intended forcing function, but it is worth lining up the post-promotion refresh (drop:free, keep the paid rates) as an immediate follow-up so the red window stays short. Related nit: the failure message intests/test-pricing.ts:415says "expired on 2026-10-20", which reads as if the promo ended a day early; wording it as "needs recheck after 2026-10-20 (promo ends 2026-10-21T10:45Z)" would avoid that confusion. - P3 —
stealth/cat-jam:freeis still insrc/commandcode-catalog.ts(the pinned CLI1.84.0still ships its metadata: reasoning, efforts, 131072 output) while its price entry is gone.index.ts:179falls back toZERO_MODEL_COST, so the model renders as $0. Harmless here since it was free, but it is the general silent-zero path; the unknown-price warnings from #159 will cover it once merged. - P3 — both new MiniMax variants keep the conservative defaults (text-only, non-reasoning, 65 536 output) even though the pricing page advertises
text+vision+reasoningand a 1M context. Consistent with the repo rule of following the published CLI metadata, same as Beam, but worth stating in the release note. - P3 — the pricing fixture is generated from
src/pricing.ts, sodeepEqual(MODEL_COSTS, expected)stays self-referential; the namedassertCostcalls andcheck:commandcode-pricingremain the real drift detectors (unchanged from my earlier note). - P3 — smaller residue, none blocking: the model-id fixture lost its dated freshness pin (
tests/test-pricing.ts:58-60now checks ISO well-formedness only), the new deal-matching unit test coversbeam-freebut not the expiringminimax-m3-1-flash-freeshape, and the twoUnreleasedbullets end up ordered 2026-10-20 → 1.83.0 → 1.84.0, which is neither newest-first nor oldest-first.
Method note: a fresh reviewer was launched in an independent context on this head, but its session came up with read-only tooling, so it reviewed the sources and the recorded command output rather than re-running the suites. Every suite result quoted above was produced here at the exact head.
|
While this PR was waiting, upstream moved quite a bit (CLI 1. Catalog pin 2. Metadata the CLI now publishes for exactly the models your revision called "no metadata yet". These stop being hypothetical conservative defaults:
3. Missing prices that keep
4. Your MiniMax free entry needs a refresh: the promotion was extended. It now expires 2026-10-22T10:45:00Z on the pricing page (it was 5. With 1.87.0 plus those five price entries, both checks go green in one shot ( |
|
Time-boxed re-check at
Suggested single push, after 10:45 UTC today so both free promotions are settled:
That should leave Why the timing matters here: |
|
Follow-up at 19:00 UTC — the delta actually shrank: both free promotions ended while we were waiting. The live API is now 94 models, not 96:
So the remaining work is smaller than this morning's list:
At If you'd rather not spend another cycle on it, say the word and I'll push that single commit to your branch myself, keeping your authorship and your Beam/MiniMax/Ornith pricing work untouched. Otherwise ping me when it lands: I'll re-verify the exact head, re-publish the approval, and take it straight into the 0.7.9 release. |
pierreraby
left a comment
There was a problem hiding this comment.
Approved at 22720fecac192f2414c7f82de128b422a987b5ac. Verified independently in a detached worktree on that exact commit (npm ci --ignore-scripts), nothing modified locally:
npm test: rc=0, 63 suites green, including[pi-local] PASSand[omp-compat] PASS.check:commandcode-catalog: Current againstcommand-code@1.87.2(all capability rows unchanged).check:commandcode-pricing: PASS, 94 models, no drift, no unused local price.- Fixtures: 94 ids,
fetchedAt2026-10-22T20:38:00Z;TEMPORARY_PRICINGempty with the explanatory comment;PRICING_LAST_VERIFIED2026-10-22. - The
1.87.1→1.87.2capability delta is exactly what the changelog says: retired free-model metadata only (minimax/minimax-m3.1-flash:free,ornith/ornith-1.5-122b:free), paid variants untouched.
Re-checked the timing facts independently: ornith/ornith-1.5-122b:free left the API at 2026-10-22T06:00:00Z and minimax/minimax-m3.1-flash:free at 10:45:00Z, so the API now advertises 94 models with only reflection/beam:free and stealth/glyph-cluster:free remaining free; paid MiniMax M3.1 Flash and Ornith rates still match the official page. The removal of both :free entries and of the dated guards is correct, and the test changes only follow the data.
No blocking findings from me. Approving so this can go into the 0.7.9 patch release, which also carries the lifecycle-script guard from #157 and the 1.83/1.87 catalog work.
Summary
command-code@1.87.2. This version removes image/reasoning metadata for retired free MiniMax Flash and Ornith variants, plus selectable efforts for free MiniMax Flash.low/medium/high/xhigh/maxefforts; Beam supports reasoning with the same efforts and a 131,072-token output limit; Ornith 1.5 122B supports image input and reasoning without selectable efforts.reflection/beam:free, free while capacity lasts with no published expiry.TEMPORARY_PRICINGis now empty.The capability snapshot now also excludes both retired free variants. Regression tests verify that their image, reasoning, effort, and output-limit metadata are absent.
Automated sync overlap
The CLI snapshot changes overlap with automated catalog synchronization, including PR #168. Pricing, fixtures, tests, and changelog remain manually reviewed additions.
Verification
Verified on the latest revision,
22720fe:npm run check:commandcode-catalog: current againstcommand-code@1.87.2(72 image-capable models, 81 reasoning models, 65 models with selectable efforts, 10 model-specific output limits).npm run check:commandcode-pricing: PASS, 94 models, no pricing drift.npm test: PASS, including real Pi and Oh My Pi integration against mock APIs..pi/tasks/harness artifacts excluded).git diff --check: PASS.