fix(web): align /internal/* result cache with the public routes (#143) - #144
fix(web): align /internal/* result cache with the public routes (#143)#144lukaso-bot wants to merge 22 commits into
Conversation
Preview deployed
Federated GitLab lookups (freedesktop / GNOME) degrade to the "use the CLI" card — the Anubis relay is off in preview. GitHub lookups, permalinks, and OG render work once |
|
Status on this PR's red gate — none of it is the cache fix. Three jobs failed on head 1. Two were a GitHub infra outage, not this branch. GitHub is degraded right now more broadly — the GraphQL API is returning 503s to 2. So this PR needs no code change for either. It goes green once #146 merges and |
Addresses the three review findings on #144. All three are defects in what this PR introduced — it made /internal share the public routes' cache slot, so it has to share the policy and the failure modes that govern it. - Cache faults are never fatal. Only the FIRST get was guarded; the inner get and the put sat inside the try whose catch returns 503, which web-og renders as the neutral placeholder — the #143 symptom, from a successful lookup. A neverFatal() wrapper degrades any Cache API refusal to "served, just not cached". - /internal now resolves through resolveLookup, the same resolver the public routes use: per-state hard TTLs (30d terminal / 24h pending / 60s partial) instead of a flat 30 minutes, getEntry()+isFresh() instead of a bare read, and the negative back-off for free. A cold OG render no longer downgrades a 30-day terminal slot to 30 minutes, and no longer serves a 60-second partial for half an hour after the public page has moved on. - Preview sets its own PUBLIC_BASE_URL. PROD_HOST is committed in [env.preview.vars] too, so preview keyed /internal on a host it does not serve while its public routes keyed on the preview origin — #143, unfixed in the one environment OG changes get reviewed in. Guards mutation-proved: all 7 new tests were RED on the pre-fix code (503 on a refused write, 503 on a throwing read, max-age=1800 where the policy says 2592000 and 60, a 10-minute-old pending answer served without revalidation, no negative marker, no preview PUBLIC_BASE_URL), 14/14 green after. Full gate green: 599 tests, typecheck, lint, build, deploy-config.
|
Review round addressed in 4d2a239 — all three threads applied and resolved.
7 new tests, every one RED on the pre-fix code and green after; full local gate green (599 tests, typecheck, lint, build, deploy-config dry run). The |
…cache origin Two findings from the review round on #144, both #143 reintroductions through the new /internal cache path. 1. Sharing the public routes' cache policy also shared their WAIT. resolveLookup revalidates a pending answer after 5 minutes and a partial after 60 seconds, and web-og awaits /internal with no timeout — so a merely-stale entry put findRelease's 24s soft deadline back on the crawler's critical path, and a blown deadline hands the crawler the neutral placeholder at max-age=60. That is the #143 outcome reached from a stale slot instead of a cold one. resolveLookup gains an opt-in `revalidate` callback: a stale answer is served immediately and the refresh runs via executionCtx.waitUntil. The public HTML routes omit it and keep the blocking behaviour, so their semantics are unchanged. A genuinely cold slot still blocks — there is nothing to serve — but it write-backs, so it is cold at most once. 2. cacheOrigin concatenated a scheme onto PROD_HOST unconditionally. PROD_HOST is shared with isProdRequest(), which documents itself as tolerant of a value written WITH a scheme; `new URL('https://https://host')` does not throw, it yields origin `https://https`, so every entry would key on a non-routable host the Cache API drops — silently, with neverFatal swallowing it. The reverse slip was worse: a scheme-less PUBLIC_BASE_URL made `new Request()` throw OUTSIDE neverFatal, turning a computed answer into a 503 → placeholder. Both spellings now normalise through URL().origin. Mutation-tested — each guard was watched failing on the defect it names: - drop `revalidate:` → both stale-while-revalidate tests time out (the render waits on an upstream lookup that never resolves). - restore the naive concatenation → the PROD_HOST-with-scheme test reads 'MISSED-THE-PUBLIC-SLOT' instead of the seeded 'v4.16.0' (proving the entry landed in a different namespace), and the scheme-less PUBLIC_BASE_URL test gets 500 instead of 200. Tests use a distinct SHA each so a hanging test cannot poison the next through singleFlight's module-level in-flight map. Full gate green: 296 passed | 6 skipped (web), typecheck clean.
Review round 2 addressed —
|
| mutation | result |
|---|---|
remove revalidate: from /internal |
serves a stale pending answer without waiting and serves a stale PARTIAL without waiting both time out — the render waits on an upstream that never resolves |
| restore the naive scheme concatenation | PROD_HOST written WITH a scheme reads MISSED-THE-PUBLIC-SLOT instead of the seeded v4.16.0 (proving the entry landed in a different namespace, not just that a string differed); PUBLIC_BASE_URL written WITHOUT one gets 500 instead of 200 |
Each test uses a distinct SHA, because singleFlight keys a module-level map that outlives a test —
a hanging test would otherwise poison the next one through it (that artifact cost me a confusing
first mutation run showing 4 failures where only 2 were real).
Gate: 296 passed | 6 skipped (web), full pnpm -r test green, typecheck clean. The red osv dependency scan is the main-wide #145 (3 High in the lockfile), which #146 clears — unrelated to
this diff.
Split out, not dropped
- OG cards key on a 7-char SHA while permalinks key on the full SHA — the first unfurl never reuses the warm slot #147 — commit OG cards key on a 7-char SHA while permalinks key on the full SHA, so the
primary search → view → share flow doesn't reuse the warm public slot on the first unfurl. The fix
touchesog-meta.tsx/result.tsx(untouched here) and changes the public key namespace — a
cache-invalidation event that deserves its own review. - Preview URL drift (the
wrangler.tomlthread) — the committed subdomain is verified
(/healthz200, and it matches the URLpreview.ymlposts here), so I left the value alone. The
drift guard belongs inpreview.yml, which should write its discovered URL into the var;
backlogged. I deliberately did not add a test asserting the literal host — it can't fail for
the reason that matters, so it would read as coverage while proving nothing.
⚠️ The preview could not actually exercise this change — see #148
I exercised the preview deploy as required and found it renders the neutral card for every
input: a real commit gives 14348 B and a nonsense SHA gives 14244 B, both under a second, versus
28899 B in production. og-preview isn't reaching web-preview's /internal at all (likely
INTERNAL_SECRET unset on the preview Workers — preview.yml documents it as a manual per-env
step, and isServiceBinding correctly fails closed). Filed as #148 with the exact commands;
it needs a human, since the loop doesn't touch secrets.
So: this PR is verified by the gate and by mutation testing, not by the preview. Worth knowing
before merge, given the change is specifically about the OG path.
|
Preview exercised on That covers the actual risk this PR carries. The change is in What the preview cannot cover: Gate status unchanged: the single red check is |
The one remaining red is not this PR's
Evidence — this branch's dependency files are byte-identical to main's: The three High advisories the gate trips on are the ones tracked in #145: #146 raises the override floors that clear all three. I re-scanned #146's Only four Merge order
The catch is that GitHub does not re-run PR checks by itself when the base branch Everything else here is green, and the preview was exercised on |
Review round 2 addressed — head is now
|
| # | Finding | Disposition |
|---|---|---|
internal.ts:75 |
originOf can return the literal string "null" |
Applied + regression test |
wrangler.toml:130 |
The alignment can't be exercised in preview | Applied (comment corrected) |
internal.ts:180 |
Crawler-written :neg marker degrades the human permalink |
Declined — deliberate, now documented |
Guard proof (the opaque-origin fix)
Written test-first and watched fail on the defect in its own title, not an adjacent one:
- Mutation:
PUBLIC_BASE_URL=file:///srv/web. It contains//, so it skips the scheme-prefix branch,new URL()parses it, and.originis the string"null"— non-null, so it satisfies??and reachesnew Request(cacheOrigin(env, req)), which is outsideneverFatal. - Assertion that fired:
expect(res.status).toBe(200)→AssertionError: expected 500 to be 200. That 500 isapp.onErrorturning a computable OG lookup into web-og's neutral placeholder — the same class as the scheme-lessPUBLIC_BASE_URLslip already fixed here. - After the guard: 200, falls through to
PROD_HOST's public slot,findReleasenever called.
On the preview claim
I checked Cloudflare's current Cache API docs rather than trusting my priors. Cache operations are functional only for Workers on custom domains. Our preview is *.workers.dev, so it can never warm a slot and therefore can never validate this alignment. The comment claiming otherwise is gone; the PUBLIC_BASE_URL var stays (it keeps preview off the prod key namespace and drives canonical URLs). The alignment is proven by the unit tests and takes effect on the prod custom domain.
Verification on df20483
- Full gate local:
lint(exit 0),typecheck,pnpm -r test— 601 tests pass. - CI on this head: all
testlegs, a11y, deploy-config dry run, gitleaks, shell+workflow lint pass. - Preview exercised (
released-web-preview.lukaso.workers.dev):/healthz,/,/how-it-works200; permalinkhonojs/hono@f82aba8→ v4.12.11; badge renders. Note per the above that preview cannot exercise the cache behaviour itself — this proves the code runs, not that slots warm.
The one remaining red is still not this PR's
osv dependency scan fails on main's lockfile, not on anything here — this branch's package.json and pnpm-lock.yaml are byte-identical to main's. That's #145, and #146 is the fix.
Merge order: #146 first. Then gh run rerun this PR rather than rebasing — ci.yml uses a bare actions/checkout on pull_request, so the checks already scan refs/pull/144/merge (this branch merged into main's tip); #146 landing clears the scan with no commit here, but GitHub won't re-run on its own when the base moves.
… rationales Round 15 review of #144. Three findings, all in this diff's own code or its own docs. 1. `writtenNoEarlierThan`'s missing-stamp fallback went the OPPOSITE way from the direction its header claimed. `cache.getEntry` reports `ageSeconds: 0` for an entry with no `x-cached-at`, so an unstamped pair evaluated `0 <= 0` -> true and the marker vouched for the entry unconditionally. The fallback is removed: an unprovable ordering now recomputes, as documented. 2. The `bypassBackOffWhenUnservable` doc still said "the marker is still WRITTEN on failure". `ea541fe` added `if (!backedOff)`, which makes that false on exactly the path the flag creates. 3. The shared-TTL note explained the gallop-partial misclassification but not what this PR changes about it: sharing the key makes an UNFURL a writer of the slot, so crawler traffic can now pin badge/permalink to an unconfirmed gallop tag. Stated plainly, with the fix still scoped to #155/#159.
Round 15 — all five threads drained, and a decision this PR now needs from youThree findings applied ( But draining threads is not the same as this PR being ready, and I want to put the real state in front of you rather than leave it in a thread. Where this PR actually isIt opened as "align The thing that should decide itThe finding on I could not settle it. Cloudflare's Cache API docs list where the Cache API has no effect — dashboard, Playground, Workers behind Access — and say custom-domain Workers have functional caching. Service bindings are not mentioned either way. The unit tests use an in-memory store and pass under both causes; Three ways to go, and what I'd pickA — Merge as-is, then run the one check that settles it. (My recommendation.) #143 is a real defect live in production right now: every cold OG unfurl serves the neutral placeholder, and that placeholder is what Slack and X have cached. This PR is green and every guard in it is mutation-proved. Merge it, then request a cold OG URL twice: Second response real card + B — Land less. If 2280 lines is more than you want to merge on an unconfirmed diagnosis, I can split: the key alignment alone (the original fix) in one PR, the SWR/partial/back-off machinery in a second. Honest cost: a few more rounds, and #143 stays broken in production while we do it. C — Do #162 first. Have web-og call the binding on I'd take A: ship the fix for the live bug, then spend one command learning how much of it we get to delete. I'm not merging anything — this is your call. Round 15 changes in detailApplied — The second is seeded so the age comparison gives the wrong answer by a different route (entry unstamped at 5s, marker stamped at 0s → Applied — two rationales that no longer matched the code. The Declined — the partial throttle's re-read. |
… test rationale Two comment-only corrections from review round 16. No behaviour change. - `routes/internal.ts`: the doc block claimed the `/internal` key "MUST match the public permalink routes' exactly" without noting that this holds for the issue and PR routes only. On the commit route `og-meta.tsx` builds the `og:image` URL from `shortSha()` (7 chars) while `result.tsx` keys the permalink on the full 40, so a commit unfurl still misses the slot the permalink warmed. Record the caveat and point at #147, so the function is not read as having closed #143 for commit links. - `test/cache.test.ts`: the rationale said an unstamped pair makes an ordering test "fall back to ages". `writtenNoEarlierThan` does the opposite — it returns false when either stamp is null, so the pair is always recomputed, never served. State what ships.
Round 17 review, packages/web/src/routes/internal.ts:58. `cacheOrigin` passes its two CONFIGURED arms through `originOf`, which rejects a single-label host precisely because `https://web` is what the Cache API silently declines — but the request-origin fallback skips that check, and for a real Service Binding that origin IS `https://web`. So if PROD_HOST is dropped from [vars], blanked in the dashboard, or a new [env.*] ships that the committed wrangler.toml does not describe, every /internal read misses and every write no-ops. `neverFatal` then renders that as "served, just not cached": #143 is back with no error, no log and no metric — the silence that made it look green for weeks. Behaviour is unchanged; the relapse just stops being invisible. The wrangler.toml guard in internal-cache-origin.test.ts covers the committed file, and this covers the config it cannot see. Mutation evidence (all three assertions fail on a concrete input): - remove the warn -> "warns when it falls back to a request origin the Cache API will decline" fails, expected '' to contain 'https://web' - warn unconditionally -> "stays silent when the request-origin fallback is itself routable" fails - warn above the configured early return -> both "stays silent" tests fail pnpm -r test 429 passed, pnpm -r typecheck, pnpm lint clean.
Round 18 review, packages/web/src/routes/internal.ts:73. The warning added in 70a1640 only fires when NEITHER var is set. A var that IS set but rejected by `originOf` is indistinguishable from unset: the `??` chain falls straight through to the next arm, `configured` comes back truthy, and the fallback branch is never reached. The reviewer's own scenario does not hold — a fall-through to a valid PROD_HOST lands on `https://released.blabberate.com`, which this Worker does serve, so the cache works. But the finding survives that correction, because the real hazard is PREVIEW, not prod: PUBLIC_BASE_URL exists specifically so preview does not key on production's origin, and PROD_HOST is committed in [env.preview.vars] too (it gates analytics). Mistype PUBLIC_BASE_URL in the dashboard — where the wrangler.toml guard cannot see it — and `originOf` discards it, PROD_HOST answers instead, and the preview Worker writes every /internal entry onto the PRODUCTION origin. Silently, with a perfectly routable origin hiding the fault. Behaviour is unchanged; the discarded override just stops being invisible. Mutation evidence (every assertion fails on a concrete input): - drop the `raw &&` present-check -> "stays silent when a routable origin IS configured" fails (expected 1 to be +0) AND "stays silent when the request-origin fallback is itself routable" fails (expected 2 to be +0) - check only PUBLIC_BASE_URL -> "warns when PROD_HOST is set but rejected" fails, expected '' to contain 'PROD_HOST' - before the fix, both new tests were red on the real defect: expected '' to contain 'PUBLIC_BASE_URL' / 'PROD_HOST' pnpm test 431 passed, pnpm -r typecheck, pnpm lint clean.
Fixes #143. Regression of #53.
What was broken
Every OG card that wasn't already in the edge cache rendered the neutral
placeholder. Only the URLs the liveness probe re-requests each cycle came back
real, which is why this looked green for weeks. A crawler unfurls a link once,
so the placeholder is what got cached in Slack and X.
web-ogfetches result JSON fromwebover the Service Binding athttps://web/internal/.... Two independent misalignments made that lookup unableto use the result cache:
makeWorkerCache(req)derives the Cache API key URL from theincoming request, so
/internal/*keyed onhttps://web/__cache__/.... Thatis a different namespace from the public permalink routes'
(
https://released.blabberate.com/__cache__/...), andwebis anon-routable hostname, which the Cache API silently declines to store — the
same class of bug
cache.ts's own header note records forcache.invalid.So the OG path could neither reuse a warm public entry nor persist its own.
That is why Cold-cache OG unfurls serve the placeholder again (#53 regression) — probe is blind because it re-warms its own fixtures #143 saw three consecutive cold requests all return the
placeholder: it never self-healed.
result.tsx,issue.tsx,pr.tsx) key onfive parts —
('res', host/path, id, 'cull', 'nopre')— and spell idsissue#<n>/pr#<n>./internalused a three-part key withissue:<n>/pr:<n>. Even with the origin fixed, that can never land on a slot a publichit warms.
This is the "cache-key alignment" item the backlog had been carrying, and it was
the whole of the defect.
The fix
/internal/*on the canonical public origin:PUBLIC_BASE_URL, else thecommitted
PROD_HOSTvar, else the request's own origin (wrangler devandtests, where neither is set). No config change —
PROD_HOSTis already set inboth envs.
strict: false, includePrereleases: falseexplicitly so the slot written is the one a defaultpermalink hit reads back.
is deliberately no longer this request's origin, so a Cache API refusal must
not become a placeholder.
The result cache holds a pure function of upstream state (which release first
contains X), so sharing slots between the public and OG paths — and across
prod/preview — is safe.
Why the existing tests missed it
Every pre-existing
/internal/*test called the route with a public-lookinghttps://released.example/internal/.... Production calls it withhttps://web/internal/.... The tests never exercised the URL shape that breaks.The new
packages/web/test/internal-cache-origin.test.tscalls it with the realproduction shape throughout.
Mutation evidence
The guard was written first and run against the unfixed code. All 7 assertions
went red on #143 itself:
The decisive one — the cold-lookup write-back:
Array(1)is the point: the cold lookup wrote exactly one entry, and it wasunder
https://web— the namespace the real Cache API drops. After the fix, 7/7pass, and the same assertion pins that nothing lands under
https://web/at all.Each assertion has a concrete failing input: swap
issue#back toissue:, dropeither option suffix, or drop the origin override, and a named test goes red.
The five pre-existing
/internal/*tests inintegration.test.tswere seedingthe old three-part key; they are updated to the public scheme (that they had to
change is itself the bug).
Gate
pnpm -r test592 passed,pnpm -r typecheck,pnpm -r build,pnpm lintallclean locally.
osv dependency scanwill likely fail this PR, for a reason unrelatedto this change.
pnpm-lock.yamlis untouched here (git diff origin/main -- pnpm-lock.yamlis empty), but newly published advisories now flag 3 Highvulnerabilities in the existing tree —
js-yaml4.3.0,nanoid3.3.16,undici7.28.0 — and
osv-check.shgates High/Critical on PRs. This affects every openPR equally, not just this one. Filed separately.
Verifying after merge
Deploy is push-to-main. Once live, the check is a cold commit (one never
requested before):
GET og.released.blabberate.com/r/<owner>/<repo>/c/<sha>.pngfor a fresh sha.cache-control: max-age=86400. The placeholder'smax-age=60is the reliable tell that it regressed.Note the liveness probe cannot see this: it re-requests a fixed set of OG URLs
each cycle, which keeps exactly those warm. A probe change for cold inputs is
tracked separately in #143's last section.
Round 3 — scope correction
Two threads on
4cdb212. One was a defect in this diff and is fixed infff6404; one was pre-existing and is split out.Fixed — shared back-off could hand the crawler a placeholder (
fff6404). Aligning the key also aligned the<key>:negback-off marker, which public page views write. A human's failed page view could therefore 503 the next OG render without ever callingfindRelease, and the crawler caches that placeholder for good — #143 again, through the alignment meant to fix it.resolveLookupgains an opt-inbypassBackOffWhenCold, set only by/internal: with a prior, stale-serve still wins and the down host is untouched; only the cold case attempts, because there the alternative is permanent. Mutation-proven — against4cdb212the new test failsexpected 503 to be 200, the exact placeholder path.Corrected claim. My round-2 comment said the cold path means an unfurl is "cold at most once". That is only true for inputs where
findReleasereturns. It throwsNotYetReleasedErrorfor a commit that is merged but not yet in a release — the most likely thing to be freshly shared — and that still 503s to the placeholder, uncached, on every unfurl.Not fixed here, filed as #150. That
not_yet503 is unchanged by this PR:main503s on it too, via the oldcatch (err). It is a separate defect on a separate path (theweb↔web-ogJSON contract) whose fix needs its own decisions — what shape the 200 carries and what TTL a pending "not yet" answer gets. It belongs in a PR that is about it, not a third topic under this title.Note: the failing
osv dependency scanon this PR is #145 (3 High advisories inmain's lockfile blocking every PR), fixed by #146 — not a defect in this branch.Round 4 — all three findings were in the SWR path, not the alignment
Three threads on
df20483, all on the stale-while-revalidate machinery added in4cdb212/fff6404. All three were real defects in this diff (onmain,/internaldid a barecache.getwith a flat 30-minuteput, so none of thesepaths existed). Fixed in
11dc61c.1. SWR had no upper staleness bound.
prior && revalidatefired for anyentry past the 5-minute freshness window, but
HARD_TTL_PENDINGis 24h — so anunfurl could be handed a 23h-old "not yet released" answer, which web-og then
pins for another 24h (
renderImageisresult ? longCache : shortCache). Thebackground refresh fixes the slot but cannot invalidate a PNG already rendered.
Now bounded by
SWR_MAX_STALE = 30 * 60; past it we block. 30 minutes is what/internalallowed as a flat TTL before it shared this slot, so the bound is byconstruction never worse than the code it replaced.
2. The background refresh owned the
singleFlightentry. The recursive callreached
singleFlight(key, …), whose module-level entry is cleared only in theloader's
finally. UnderwaitUntil, workerd can tear the IoContext down beforethe subrequest settles: the promise never settles, the
finallynever runs, andevery later request in that isolate on that key — a human on the permalink,
badge.tson the samecull/noprekey — joins a dead promise. The refresh nowpasses
coalesce: falseand runs the loader directly.3. The back-off bypass comment claimed a throttle that doesn't exist.
singleFlightcollapses only concurrent calls within one isolate, so sequentialand cross-colo unfurls each run a full lookup against a down host. Comment
corrected to state the real cost; the gating alternative is declined on the
thread (it moves which unfurls get a permanent placeholder rather than
removing them).
Mutation evidence
Each guard was reverted individually against
11dc61cand reddens on its owndefect:
The second is the failure mode itself: with the mutation in place the follow-up
request hangs on the dead flight instead of computing.
Gate:
pnpm test611 passed,pnpm -r typecheck,pnpm lintclean.Split out, not fixed here
The residual behind finding 1 is pre-existing and in another package: web-og
long-caches any non-null result for 24h, so even a perfectly fresh "not yet
released" card can't flip — the OG analogue of the badge invariant this repo
already states. Filed as #151 (with the one-line fix, the
partialcase, theoverlap with #141, and the guard it needs). Not pulled in here: it is a different
file, a different Worker, and #141 is already open against those same lines.
osv dependency scanis still red for #145's reason (3 High advisories inmain's lockfile, fixed by #146), not for anything in this branch.Round 5 (
2873d1c)Two threads, both in this PR's own diff.
1. The SWR background refresh was uncoalesced and unconditional. Round 4's
coalesce: falsekept awaitUntiltask from owning thesingleFlightentry, but italso dropped the refresh out of coalescing entirely — and the branch fires on every
request in the stale window. One link unfurled by four platforms in the same second in
one colo ran four full
findReleasetraversals against the same repo on the sharedtoken.
Fixed with a second map (
backgroundFlight) that foreground callers never join:background refreshes collapse onto each other, and no live request can join a task the
runtime may tear down. The round-4 poisoning guard passes unchanged.
(Note: "join an existing flight without registering" — the shape the review suggested —
does not fix this case. All four requests are stale hits, so there is no foreground
flight to join and all four still run.)
2. The guard file exercised only
'a'.repeat(40). Production sends a 7-char sha(
ogImageUrlForCommit→shortSha). Two tests added on that shape.Mutation evidence:
The remaining key-part misalignment (
sha:<7>vs the full sha the public route keys on)is #147, deliberately not folded in: its fix is in files this PR does not touch and
changes the public routes' key namespace.
Gate: 613 tests, typecheck, lint clean. The red
osv dependency scanis inherited frommain's lockfile (#145) and clears when #146 merges.Round 8 —
5896333Four findings from the independent review of
fd09b1b, all in code this PR introduces.Each guard was mutation-proved against the defect in its own title, not a nearby one.
1. The computed-partial 503 was never recorded, so it wasn't throttled
Round 7 made
/internal503 a computedpartialrather than pin a wrong "not yetreleased" card. Correct — but
resolveLookupwrites that partial to the slot and thenno read path accepts it (fresh, SWR and back-off exits are all gated on
!unpinnable),so
run()'s re-read fell straight through toload(). On a repo that reliably blows the24s soft deadline that is a full traversal per unfurl on the shared token, where the
flat 30-minute TTL this route replaced made zero upstream calls.
Fixed in
unpinnable()rather than with a new marker: a partial inside its ownHARD_TTL_PARTIALis handed back and the route still 503s it, so the refusal costs acache read instead of a
findRelease. Reusing the entry the resolver already wrote meansa real answer landing in the slot inside that window simply overwrites it and wins.
2. A gallop-only
partialwas treated as terminal and pinned for 30 daysThe pin bound exempted any entry carrying a
firstRelease. Butfind-release.ts:293-305returns a
partialwhosefirstReleaseis the gallop hit — the bisect that wouldconfirm no earlier release contains the commit is exactly what the deadline cut short
("not necessarily the earliest",
find-release.ts:724). The result card renders thatcaveat; an OG card cannot, and web-og pins the bare tag for 24h. No partial is servable to
a pinning consumer now, either shape.
The underlying misclassification —
hardTtlFor()/isFresh()both testfirstReleasebefore
partial, so the shape is stored 30 days and reported fresh forever — is onmain,predates this PR, and affects the public routes too. Filed as #155, deliberately
not widened into this PR.
3. The request-origin fallback could silently reinstate #143
With both
PUBLIC_BASE_URLandPROD_HOSTunset,cacheOriginreturns the requestorigin — for a real Service Binding, the non-routable
https://web. Two guards:originOf()now rejects a single-label host (localhostexcepted, forwrangler dev),and a new wrangler.toml suite enumerates
[vars]+ every[env.*.vars]and asserts eachsets one of the two, routably. The suite enumerates rather than hardcodes, so a new
named env is checked the day it is added.
4.
backgroundFlightentries never expiredThe map is module-level, so an entry registered under request A's IoContext is handed to
request B; when workerd cancels A's context the promise never settles and the loader's
finally— the only thing that clears the entry — never runs. Every later refresh forthat key joins the dead promise, so background revalidation is dead for the isolate's
lifetime and entries accumulate one per key. Entries now carry
startedAtand expire at30s (
findRelease's hard deadline is 28s), and thefinallyonly clears the entry if itis still the live one — otherwise a late-settling abandoned promise would evict its own
replacement.
Mutation evidence
Every new guard has a passing complement so it cannot hold by rejecting everything:
recomputes once the recorded partial ages out of its 60-second window,serves a real answer that landed in the slot inside the partial window,...but a TERMINAL answer of the same age is still served, never recomputed,still joins a refresh that is merely SLOW, inside the deadline,still accepts localhost, which wrangler dev really serves on, anddeclares at least one env to check, so this suite cannot pass vacuously.Two round-6/7 tests in
resolve.test.tsasserted the old recompute-always behaviour andwere rewritten, not deleted — the property they protected (a partial is never pinned)
is unchanged; only where the refusal happens moved from the resolver to the route.
Gate on
5896333: typecheck clean across all 4 packages, lint clean, fullpnpm testgreen.Round 10 —
317ca2b— scope stated plainly, two costs written downThree findings, all confirmed. None was a defect in this PR's diff, so the diff did not
grow: the change here is doc-only (+19 lines of comment, no behaviour change).
What this PR does NOT fix — say it before the title implies otherwise
The commit route — the product's most common permalink — is still misaligned, so
#143's user-visible symptom survives there.
og-meta.tsx:115truncates the sha to7 characters for the image URL, while
index.tsputs the full 40 in the permalink("Use the FULL SHA in the permalink"). So:
/r/honojs/hono/c/<40-char sha>→result.tsxwarmscacheKey('res', 'github.com/honojs/hono', 'sha:<40>', 'cull', 'nopre').../c/<7-char>.png→ web-og calls/internal/result/honojs/hono/<7-char>→ this PR keys on
sha:<7>Different digest, so the slot the permalink just warmed is invisible and the unfurl still
pays a full
findRelease. Verified against production: the bot og:image for that permalinkis
https://og.released.blabberate.com/r/honojs/hono/c/f82aba8.png?v=og.v1.The issue and PR routes ARE fully aligned by this PR. The commit route gets the origin
fix and the key-part fix but not sha-length alignment — that is #147, whose fix changes
the public routes' key namespace in files this PR does not touch. The title's "align" should
be read as "align the origin, the key parts and the id spelling", not "close #143 on every
link shape".
Two costs now written into the code, not left implicit
Both are consequences of correctness decisions this PR makes deliberately. Neither is fixed
here; each has an issue with a concrete remedy sketch.
partialmeans a deadline-heavy repo(
gitlab.gnome.org/GNOME/gimp) gets a permanently blank card.HARD_TTL_PARTIALandweb-og's placeholder
max-ageare both 60s, so the "throttled to one traversal per60s per key" note the code claimed is true but buys close to nothing against a crawler
asking once a minute: the card never converges and upstream load goes from ~0 to a full
traversal per minute, on the shared token through a
max_instances = 1relay. Thecomment now says that instead of implying the cost is bounded. Remedy (serve the gallop
hit, have web-og short-cache it) is adjacent to OG card for a not-yet-released commit is cached 24h, so it never flips when the release lands #151.
bypassBackOffWhenUnservablebypass is unconditional, and the existingrationale rebuts the obvious objection (gating on a fraction of
NEG_TTLonly moves whichunfurls get the placeholder). It does not cover the second-order effect: every bypassed
load re-stamps the marker, so it is almost never older than
NEG_TTL, and human pageviews — which do not bypass — sit on the "checking…" card for the whole outage instead of
getting a per-minute retry window. Fixing that means changing the marker's semantics, not
the bypass condition.
Gate on
317ca2b: fullpnpm testgreen — 405 tests across 4 packages, lint and gitleaksclean, osv clean of High/Critical.
Round 11 — three findings, all defects this PR introduced
Each is code this diff added, so each is fixed here rather than deferred. Each is bound to
a mutation that reproduces the specific failure claimed, run before and after.
1.
/internalcould join badge's 8-second flight and inherit its truncation.Aligning the cache key also aligned the in-isolate single-flight key.
singleFlighthandsevery joiner the first registrant's promise and runs only the owner's loader, and
badge.ts:110builds a byte-identical key forissue#N/pr#N— with a deliberately tighter8s/9s deadline against this route's 24s/28s. A badge request landing first would hand
/internalapartial, which it 503s into a pinned neutral placeholder: #143's symptom, ona link this route's own deadline answers. On
mainit could not happen —/internalkeyedon a three-part key of its own — so this PR opened it.
resolveLookupnow takes an optionalflightKey;/internalpasses its own. The cache slotstays shared (the point of the PR); the truncation no longer travels with it. Concurrent OG
unfurls still collapse into one flight.
Mutation: delete the
flightKeyline →expected 503 to be 200.2.
hostname.includes(':')can never match a port.URL.hostnameexcludes the port, sothat clause matched only a bracketed IPv6 literal — a dead assertion reading as coverage. Its
cost is not over-strictness: a dev origin like
http://app:8787(Codespaces / Docker / WSL,which the new README hunk invites) was rejected, and a rejected
PUBLIC_BASE_URLisindistinguishable from an unset one, so the
??chain fell toPROD_HOSTand split thenamespace despite the var being set correctly. Now reads
URL.port.Mutation: reinstate the colon clause →
expected 'v9.9.9' to be 'v4.2.0'(origin rejected,fell to
PROD_HOST, warm slot invisible, fresh lookup ran).3. The
wrangler.tomlguard passed on the config this PR is fixing. It assertedPUBLIC_BASE_URL ?? PROD_HOSTis set and routable — and[env.preview]already declared theproduction
PROD_HOST, because every env pins it to gate analytics. That is precisely how thepreview keyed
/internalon the prod origin while servingreleased-web-preview.*. Verifiedrather than argued: the old assertion returns
[]for the pre-fix[env.preview].Replaced by a pure
cacheOriginProblems(cfg)requiring each named env to have a routablecache origin of its own, exercised against four known-bad configs — the pre-fix
[env.preview]among them, so the guard is bound to the bug in this PR's title — plus thecommitted file. Not adopted: requiring the host to contain the env
name, which would false-positive on a custom-domain
[env.staging]; the preview-specific test keeps that assertionwhere it is genuinely true.
Two things deliberately left out, both pre-existing code this PR does not touch: badge and the
permalink pages have shared a flight key since before this PR (a much milder degrade — the
result card renders the
partialcaveat where the OG path pins a placeholder), and the TTLmisclassification of a gallop-only partial written by a public route is #155.
Gate on
5b1b72d: fullpnpm testgreen — 405 tests across 4 packages, typecheck, lint,shellcheck, actionlint, gitleaks, publint and both
wrangler deploy --dry-runchecks clean,osv clear of High/Critical.
Scope — what this does NOT close (review round 16)
Fixes #143stands, but the alignment is not uniform across the three routes andthe PR should not be read as claiming it is.
issue#/pr#id spelling as the public permalink. A crawler unfurl lands onthe slot a human page view warmed, and vice versa.
ui/og-meta.tsxbuildsthe
og:imageURL fromshortSha()(7 chars) whileresult.tsxkeys thepermalink on the full 40. So a commit unfurl does not reuse the permalink's
warm slot.
The dominant half of #143 is closed for every route including commit: the origin
bug (
https://web/is non-routable, so the Cache API silently declined to store)is why the OG path could never persist its own entry and never self-healed —
three consecutive cold requests all returned the placeholder. That is gone.
What survives on the commit route is narrower: the first unfurl pays one
findReleaseinstead of reading the human's warm slot, then persists normally.Closing that needs a change to the public routes' key namespace, which is
tracked as #147 and deliberately not attempted here.
Adjacent findings raised in review and tracked separately rather than absorbed
into this diff: #159/#155 (a
partialcarrying afirstReleaseisclassified terminal by
hardTtlFor/isFresh, so it is cached 30 days and neverrevalidated — this PR widens reachability of that pre-existing defect to
crawler traffic), #156 (a deadline-heavy repo gets no card while
/internal503s on
partial), #157 (the negative back-off bypass is unconditional onthis path), #160 (
ResultCardignoresresult.partial).Round 17 (
5bc4e13) — 1 applied, 2 recorded as merge-decision factsThree threads. One was a defect in this diff and is fixed in
70a1640; two areexplicitly "not asking to widen the diff" and are recorded below instead, because
what they ask for is that the merge decision know them.
Applied — the cache-origin fallback could relapse into #143 silently (
70a1640).cacheOriginpasses its two configured arms throughoriginOf(), which rejects asingle-label host precisely because
https://webis what the Cache API declines. Therequest-origin fallback skipped that check — and for a real Service Binding that origin
is
https://web. So ifPROD_HOSTis dropped from[vars], blanked in the dashboard,or a new
[env.*]ships that the committedwrangler.tomldoes not describe, every/internalread misses and every write no-ops, andneverFatalrenders it as "served,just not cached" — #143 back with no error, no log, no metric. The
cacheOriginProblemssuite guards the committed file; it cannot see a dashboard-set var. Now a
console.warnmakes it visible in
wrangler tail. Behaviour is unchanged.Mutation evidence — each of the three assertions has a concrete input that reddens it:
The first is the failure mode itself: silence is the bug.
Merge-decision facts (no code change, deliberately)
Both of these are real, both are already filed, and absorbing either would change a
policy this PR is not about — the diff-growth trap. Recording them here so the button
is pressed knowingly:
Merge order is
#144 → #158 → #156, and#144alone regresses deadline-heavyrepos (
internal.ts:318, tracked as Deadline-heavy repos get a permanently blank OG card and one full traversal per minute after #144 #156). This branch refuses everypartialwith a 503 — correct, since an OG card cannot render the best-effort caveat — but
HARD_TTL_PARTIALand the placeholder'smax-ageare both 60s, so for a repo likegitlab.gnome.org/GNOME/gimpthe card never converges and upstream load goes from~0 to one full traversal per minute through a
max_instances = 1relay. Onmainthat card at least rendered the gallop tag. fix(web-og): cache the OG card by terminality, not by result presence (#151) #158 (terminality-keyed OG cache) then
Deadline-heavy repos get a permanently blank OG card and one full traversal per minute after #144 #156 is what closes it. Merging fix(web): align /internal/* result cache with the public routes (#143) #144 without fix(web-og): cache the OG card by terminality, not by result presence (#151) #158 behind it is the state that
regresses.
This PR adds crawler unfurls as a writer of unconfirmed 30-day entries
(
resolve.ts:386, tracked as partial-with-a-gallop-hit is cached as terminal: 30-day TTL, never revalidated #155/partial-with-a-firstRelease is cached as terminal on the web + badge surfaces (30d / 24h), so a truncated traversal pins a possibly-wrong tag #159).hardTtlFor()andisFresh()(
resolve.ts:54,60) both testfirstReleasebeforepartial, so a gallop-hitpartialis stored on the 30-day terminal TTL and reported fresh forever;badge.ts:141reads that same key and serves the unverified tag atmax-age=86400with nowhere to put a caveat. The shape is pre-existing —
badge.tsand thepermalink pages could already write it — but before this PR crawler traffic could
not, and an unfurl needs no human in the loop. partial-with-a-gallop-hit is cached as terminal: 30-day TTL, never revalidated #155 owns the
hardTtlFor/isFreshfix.
Not a fix(web): align /internal/* result cache with the public routes (#143) #144 fact at all — raised here, filed as PAT-computed results are cached in a public, auth-agnostic key namespace #164. Round 18 flagged that
/api/lookuphonours a caller-suppliedX-User-Github-Tokenyet keys the resultwith no auth component, so a private-repo answer lands in a slot the public routes
read anonymously — and argued this PR "widens it to the full card". Checked against
origin/main:issue.tsx:101,pr.tsx:86andresult.tsx:79already build thebyte-identical 5-part key and already render the full card, title included, on a
public unauthenticated permalink. So the exposure is fully present on
mainwith noOG path involved; this PR widens neither the data nor the reachability, and needs no
change for it. Real confidentiality bug, wrong PR — PAT-computed results are cached in a public, auth-agnostic key namespace #164 owns it.