Skip to content

feat(cache): add tag-based caching and revalidation helpers - #1964

Open
dinwwwh wants to merge 66 commits into
mainfrom
claude/orpc-cache-implementation-09e313
Open

dinwwwh wants to merge 66 commits into
mainfrom
claude/orpc-cache-implementation-09e313

Conversation

@dinwwwh

@dinwwwh dinwwwh commented Aug 28, 2026 •

Copy link
Copy Markdown
Member

Adds @orpc/experimental-cache: tag-based caching and revalidation for procedure output, with stale-while-revalidate, a CacheStore contract with memory, Redis, Upstash, Bun, Vercel, and Cacheable adapters plus a purge-only Cloudflare Workers Caching store, and a handler plugin that reflects cache tags into response headers for client-side revalidation or response caches in front. Concurrent callers of one key run the procedure once, and TanStack Query apps can refetch exactly the queries a mutation revalidated, with the mutation succeeding only once fresh data is in.

Resolves #1262

Features

  • cache() middleware: a hit returns the cached output without running the handler; a miss runs it once, even across concurrent callers, and stores the result. key defaults to the procedure path and input, canonically encoded so structurally equal keys share an entry; key, tags, ttl, swr, and enabled accept static values or functions of the middleware options and input, and enabled: false skips the store and the other resolvers. Durations are milliseconds, as in @orpc/experimental-lock.
  • Stale-while-revalidate: past ttl but within swr, stale output is served at once while one caller refreshes in the background. cache/waitUntil hands the refresh to runtimes that stop pending work after the response; without it, refresh failures surface as unhandled rejections rather than being swallowed.
  • revalidate({ tags }) middleware invalidates tags after a successful mutation; tags are non-empty at compile time, and resolving to null or undefined skips it.
  • CacheHandlerPlugin sets only the headers listed: orpc-cache-tag and orpc-cache-tag-invalidation for clients, cache-control and cache-tag for caches in front. cache-control uses max-age because s-maxage carries proxy-revalidate semantics that forbid stale reuse. Only the root procedure's activity is reflected, only on successful responses, and tags are percent-encoded so case-insensitive caches cannot collide distinct tags.
  • CacheLinkPlugin reads orpc-cache-tag and orpc-cache-tag-invalidation for each call, even when batched or deduplicated, for client caches to track and revalidate tagged data.
  • experimental_CacheRevalidationUtilsPlugin for TanStack Query records each query's tags and, when a mutation revalidates tags, refetches the affected active queries before the mutation succeeds, so onSuccess and mutateAsync see fresh data. Failed refetches never fail the mutation, infinite queries are tracked per loaded page, and a first load racing the mutation, such as one batched with it, is refetched rather than left stale.
  • SSR friendly: the query client always comes from TanStack's function context and tags are kept per query, so module-level links and utils are safe across requests; data hydrated from the server counts as affected until a fetch reveals its tags.
  • Stores implement getOrSet(key, fill, options) and revalidate({ tags }). Every key-value store shares BaseKeyValueCacheStore, which coalesces fills through a Locker from @orpc/experimental-lock: within the process by default, or across processes with a shared Redis locker, and a caller that times out waiting fills on its own. The Redis, Upstash, and Bun stores keep entries as JSON strings and tags as counters using single-key commands only, so they work on Redis Cluster, and share BaseRedisCacheStore and one entry format, so any two can serve the same database. The Cacheable store backs onto a Cacheable instance and its tag service. TieredCacheStore layers stores front to back, so a miss in one tier fills from the next and only the last runs the procedure, with per-tier ttl and swr caps. Every store carries one RPCJsonSerializer for keys and outputs.

Shared

  • @orpc/shared gains deepSortKeys and the cache tag header codec; @orpc/bun and @orpc/cloudflare gain their stores; @orpc/tanstack-query gains the revalidation plugin.

Testing

  • 100% coverage on @orpc/experimental-cache, with a store contract shared by every adapter, env-gated Redis and Upstash integration suites, a Bun suite, cross-adapter compatibility suites proving Redis, Upstash, and Bun share entries, counters, retention, and locks, workerd tests for the Workers store, and race and shared-locker tests.
  • The TanStack Query plugin is fully covered against real query clients, including SSR hydration, infinite pages, failed and partial refetches, and end-to-end round trips through RPCHandler and RPCLink with batching, including a mutation batched with a first load. Streaming and buffered batches, both plugin orders on each side, and GET and POST batches were also verified locally.
  • Full type-check, lint, and docs validation pass.

Docs

  • New docs/helpers/cache page with per-adapter sections and a Link Plugin section; a Cache Revalidation Plugin section on the TanStack Query page; API reference and package lists updated.

dinwwwh added 18 commits August 27, 2026 14:53
…implementation-09e313

# Conflicts:
#	README.md
#	apps/content/docs/procedure.mdx
#	packages/ai-sdk/README.md
#	packages/arktype/README.md
#	packages/bun/README.md
#	packages/client/README.md
#	packages/cloudflare/README.md
#	packages/contract/README.md
#	packages/effect/README.md
#	packages/evlog/README.md
#	packages/hibernation/README.md
#	packages/json-schema/README.md
#	packages/nest/README.md
#	packages/next/README.md
#	packages/node/README.md
#	packages/openapi/README.md
#	packages/opentelemetry/README.md
#	packages/pinia-colada/README.md
#	packages/pino/README.md
#	packages/publisher/README.md
#	packages/ratelimit/README.md
#	packages/server/README.md
#	packages/server/src/procedure-client.test.ts
#	packages/shared/README.md
#	packages/swr/README.md
#	packages/tanstack-query/README.md
#	packages/trpc/README.md
#	packages/valibot/README.md
#	packages/zod/README.md
#	pnpm-lock.yaml
@pkg-pr-new

pkg-pr-new Bot commented Aug 28, 2026 •

Copy link
Copy Markdown
More templates

@orpc/ai-sdk

npm i https://pkg.pr.new/middleapi/orpc/@orpc/ai-sdk@1964

@orpc/arktype

npm i https://pkg.pr.new/middleapi/orpc/@orpc/arktype@1964

@orpc/bun

npm i https://pkg.pr.new/middleapi/orpc/@orpc/bun@1964

@orpc/experimental-cache

npm i https://pkg.pr.new/middleapi/orpc/@orpc/experimental-cache@1964

@orpc/client

npm i https://pkg.pr.new/middleapi/orpc/@orpc/client@1964

@orpc/cloudflare

npm i https://pkg.pr.new/middleapi/orpc/@orpc/cloudflare@1964

@orpc/contract

npm i https://pkg.pr.new/middleapi/orpc/@orpc/contract@1964

@orpc/experimental-effect

npm i https://pkg.pr.new/middleapi/orpc/@orpc/experimental-effect@1964

@orpc/evlog

npm i https://pkg.pr.new/middleapi/orpc/@orpc/evlog@1964

@orpc/hibernation

npm i https://pkg.pr.new/middleapi/orpc/@orpc/hibernation@1964

@orpc/json-schema

npm i https://pkg.pr.new/middleapi/orpc/@orpc/json-schema@1964

@orpc/experimental-lock

npm i https://pkg.pr.new/middleapi/orpc/@orpc/experimental-lock@1964

@orpc/experimental-msw

npm i https://pkg.pr.new/middleapi/orpc/@orpc/experimental-msw@1964

@orpc/nest

npm i https://pkg.pr.new/middleapi/orpc/@orpc/nest@1964

@orpc/next

npm i https://pkg.pr.new/middleapi/orpc/@orpc/next@1964

@orpc/node

npm i https://pkg.pr.new/middleapi/orpc/@orpc/node@1964

@orpc/openapi

npm i https://pkg.pr.new/middleapi/orpc/@orpc/openapi@1964

@orpc/opentelemetry

npm i https://pkg.pr.new/middleapi/orpc/@orpc/opentelemetry@1964

@orpc/pinia-colada

npm i https://pkg.pr.new/middleapi/orpc/@orpc/pinia-colada@1964

@orpc/pino

npm i https://pkg.pr.new/middleapi/orpc/@orpc/pino@1964

@orpc/publisher

npm i https://pkg.pr.new/middleapi/orpc/@orpc/publisher@1964

@orpc/ratelimit

npm i https://pkg.pr.new/middleapi/orpc/@orpc/ratelimit@1964

@orpc/server

npm i https://pkg.pr.new/middleapi/orpc/@orpc/server@1964

@orpc/shared

npm i https://pkg.pr.new/middleapi/orpc/@orpc/shared@1964

@orpc/swr

npm i https://pkg.pr.new/middleapi/orpc/@orpc/swr@1964

@orpc/tanstack-query

npm i https://pkg.pr.new/middleapi/orpc/@orpc/tanstack-query@1964

@orpc/trpc

npm i https://pkg.pr.new/middleapi/orpc/@orpc/trpc@1964

@orpc/valibot

npm i https://pkg.pr.new/middleapi/orpc/@orpc/valibot@1964

@orpc/zod

npm i https://pkg.pr.new/middleapi/orpc/@orpc/zod@1964

commit: 7ecc2a4

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 28, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
orpc 7ecc2a4 Commit Preview URL

Branch Preview URL
Oct 05 2026, 09:07 AM

@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed

codspeed Bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 30 untouched benchmarks


Comparing claude/orpc-cache-implementation-09e313 (7ecc2a4) with main (7ee24a6)

Open in CodSpeed

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

One behavioral issue to resolve: a revalidation failure after a committed mutation surfaces as an error on a request whose write already succeeded. See the inline comment on revalidate.

Reviewed changes

  • @orpc/cache (new package) — cache()/revalidate() middlewares, CacheStore contract, tag-version invalidation, stale-while-revalidate, CacheHandlerPlugin header reflection, and MemoryCacheStore/RedisCacheStore/VercelCacheStore adapters.
  • @orpc/cloudflare — KVCacheStore (real KV bindings) and purge-only WorkersCacheStore, plus workerd coverage.
  • @orpc/shared — new deepSortKeys util and tests.
  • Docs/config — new docs/helpers/cache page, README/package-list updates, api-reference row, new package package.json with subpath exports, workspace wiring.

Overall this is a careful, well-tested addition. I verified the highest-risk semantics rather than taking them on faith: the tag-version technique errs on the safe side (a lost concurrency race produces a spurious miss and recompute, never a stale hit), the tag header encoding round-trips correctly under case-folding and stays consistent between the reflected cache-tag and WorkersCacheStore purge, blob/streaming outputs are guarded where they cannot be stored, and the docs call out the CDN/purge-store and per-request-shared-key caveats. Two non-blocking nits are inline.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

Comment thread packages/cache/src/middleware.ts Outdated
Comment thread packages/cache/src/middleware.ts Outdated
Comment thread packages/cache/src/middleware.ts Outdated

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

One behavior to confirm from the delta's cleanup: dropping isUncacheableOutput from the middleware also dropped the "don't record a cache check for streaming outputs" guarantee, so streaming responses are now advertised as cache activity even though every adapter ignores them. Full context inline.

Reviewed changes

This run reviews the single-commit delta e93c93e5 since the last-reviewed head 17c31c21:

  • CacheStore.revalidateTags — replaced revalidateTag(string | list) with a non-empty tag list across the interface, all four adapters, and their mocked/integration tests; stores dropped their empty-list branches and the middleware now guards with tags?.length.
  • revalidate({ tags }) API — options-object form whose tags is a Value of a non-empty tuple or undefined; resolving to undefined (or []) skips both the store call and the plugin-context recording.
  • Context keys — CacheContext moved to namespaced 'cache/store' / 'cache/waitUntil'; tests, type tests, docs, and e2e all updated, with no context.cache / context.waitUntil stragglers.
  • cache() key semantics — any provided key (string or object) is now used as given; only the omitted-key default derives [path, input]; documented as "procedures sharing a key share an entry".
  • Streaming-output drops moved into stores — isUncacheableOutput deleted; MemoryCacheStore.set now ignores ReadableStream/async-iterator outputs, matching Redis/KV/Vercel.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

`CacheStore.revalidateTag` becomes `revalidate({ tags })`, taking a non-empty
tag list, and every duration is now in seconds rather than milliseconds,
matching what Redis, Workers KV, the Vercel Runtime Cache, and `Cache-Control`
all accept. Entries without tags carry `undefined` instead of an empty array,
and stores no longer inspect output, passing it straight to their serializer.

The store and its background-work hook move to the namespaced `cache/store`
and `cache/waitUntil` context keys. A background refresh is handed over
uncaught so `cache/waitUntil` can report its failures. `revalidate` takes an
options object with a required `tags`, and a provided `key` is used as given.

The handler plugin takes header names as plain literals rather than exported
constants, sets them whatever the request method, and emits `max-age` instead
of `s-maxage`, which carries the `proxy-revalidate` semantics that would
forbid the stale reuse `stale-while-revalidate` grants.

Tag header encoding and `nowInSeconds` move to `@orpc/shared`, and each store
builds its key serializer once instead of per call.
…mplementation-09e313

# Conflicts:
#	README.md
#	packages/ai-sdk/README.md
#	packages/arktype/README.md
#	packages/bun/README.md
#	packages/client/README.md
#	packages/cloudflare/README.md
#	packages/cloudflare/package.json
#	packages/contract/README.md
#	packages/effect/README.md
#	packages/evlog/README.md
#	packages/hibernation/README.md
#	packages/json-schema/README.md
#	packages/nest/README.md
#	packages/next/README.md
#	packages/node/README.md
#	packages/openapi/README.md
#	packages/opentelemetry/README.md
#	packages/pinia-colada/README.md
#	packages/pino/README.md
#	packages/publisher/README.md
#	packages/ratelimit/README.md
#	packages/server/README.md
#	packages/shared/README.md
#	packages/swr/README.md
#	packages/tanstack-query/README.md
#	packages/trpc/README.md
#	packages/valibot/README.md
#	packages/zod/README.md
#	pnpm-lock.yaml
- RedisCacheStore and experimental_KVCacheStore take their client as the first argument
- experimental_WorkersCacheStore defaults to the cache exported by cloudflare:workers
- Redis, Upstash, and Bun stores share the shorter e:/t: key families and one envelope format
- race-condition tests for every store and the cache middleware
@dinwwwh
dinwwwh force-pushed the claude/orpc-cache-implementation-09e313 branch from e93c93e to 8a12845 Compare September 4, 2026 03:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Nested cache middleware can emit inconsistent tags and TTL metadata, and the new adapter exports lack barrel-test coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files not reviewed (1)

  • pnpm-lock.yaml: Generated file

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

packages/bun/src/index.ts:1

  • The existing Bun barrel test (packages/bun/src/index.test.ts:3-7) enumerates public exports but was not updated for BunRedisCacheStore, so this new entry point is not covered. Add it to that assertion to catch packaging/export regressions.
    packages/cloudflare/src/index.ts:6
  • The existing Cloudflare barrel test (packages/cloudflare/src/index.test.ts:3-9) enumerates public exports but omits experimental_WorkersCacheStore, leaving this new entry point unverified. Add the store to that assertion.
  • Files reviewed: 77/78 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread packages/cache/src/middleware.ts Outdated

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues in this delta — the lookup-order fix and barrel coverage read clean, with CI green on the head. One consistency nit inline.

Reviewed changes

This run reviews the delta since the prior pullfrog review at 2d6aa9aa (commits e4667ae4 and 3a729227):

  • Merged upstream main — @orpc/bun, @orpc/experimental-cache, and @orpc/cloudflare bumped to 2.0.0-beta.35 to match main's release; no other net changes.
  • Recorded stacked cache lookups in lookup order — the cache() middleware now captures its position in the plugin context before awaiting store.getOrSet and splices its record there, so stacked caches land outermost-first and the handler plugin reflects the outer cache's tags on misses and hits alike (previously inner-first, flip-flopping between miss and hit). The new regression test in middleware.test.ts fails without the fix (verified).
  • Covered the new barrel exports — BunRedisCacheStore (@orpc/bun) and experimental_WorkersCacheStore (@orpc/cloudflare) are now asserted in their packages' export tests.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

await store.revalidate({ tags })

const pluginContext = (middlewareOptions.context as CacheHandlerPluginContext)[CACHE_HANDLER_PLUGIN_CONTEXT_SYMBOL]
pluginContext?.revalidations.push({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The revalidate middleware still appends its record after next() completes, so a stacked pair records inner-first — the same reverse-order pattern this commit just fixed for caches. The revalidations JSDoc claims "in the order they ran", and the plugin's find(isRoot) will reflect the inner revalidation's tags for a stacked pair. Mirroring the lookupIndex splice would keep the two paths consistent.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No new issues in this delta — the store rename and the memory-store sweep read clean, with head CI green on all nodes.

Reviewed changes

This run covers the delta since the prior pullfrog review at 3a729227: a1dd3785 and 59beee06.

  • Renamed BunRedisCacheStore to experimental_BunRedisCacheStore — applied consistently across the @orpc/bun export, its integration tests, the cross-adapter compatibility suite, the barrel test, and the docs page. No dangling references remain, and the experimental_ prefix matches the convention already used for experimental_WorkersCacheStore.
  • Swept expired and revalidated entries from the memory store on writes — MemoryCacheStore tracks the earliest pending eviction in nextSweepAt and runs a full-map sweep on the next fill once that time (or a revalidate) has passed, re-arming the schedule from the surviving entries. The read path keeps its prior eager eviction, so the sweep is pure garbage collection and can never serve stale data. The new regression test fails against the pre-change store (verified locally), and the nextSweepAt bookkeeping checks out under both the expiring and revalidated scenarios.

The only open thread from the prior review (the revalidate record-order consistency nit at middleware.ts:136) is untouched by this window and remains so.

Pullfrog  | Fix it ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found in the BentoCache delta. This review is non-approving only because the single open Pullfrog thread on the revalidate record order (middleware.ts:136) remains open — untouched by this commit and not re-raised here.

Reviewed changes

This run reviews the delta since the prior pullfrog review at 59beee06 — the single commit 6ee2f55e:

  • Added BentoCacheStore (packages/cache/src/adapters/bentocache.ts, subpath @orpc/experimental-cache/bentocache) — drives any BentoCache setup through its own getOrSet/getOrSetForever under BentoCache's lock, maps ttl/swr to BentoCache's TTL and grace period (milliseconds), serves stale entries within grace while BentoCache refreshes them in the background, and hands a failed refresh to waitUntil — without a waiter it surfaces as an unhandled rejection, matching the documented SWR contract across stores.
  • Mapped revalidation to deleteByTag — tags delete natively, so with swr a revalidated entry is served once more while a refresh runs; a mid-fill revalidation is documented as undetected. Fill errors are unwrapped via E_FACTORY_ERROR, with non-background onFactoryError invocations guarded out so a sync miss failure isn't double-surfaced.
  • Wired the adapter in — optional peer dependency + dev dependency + lockfile for bentocache@^1.6.1, subpath export entry, a new docs page section with a setup example, and the README package-table/description sync across the monorepo.
  • Adjusted the shared store contract and the memory suite — the contract's keeps entries filled after a revalidation now sleep(1) before refilling (BentoCache's tag invalidation is millisecond-granular), and memory.test.ts fakes only Date so that shared sleep isn't blocked by fully faked timers.

I verified the load-bearing BentoCache API assumptions against the installed bentocache@1.6.1 type declarations (grace/ttl/onFactoryError accepted by both getOrSet and getOrSetForever; FactoryError.cause + isBackgroundFactory; E_FACTORY_ERROR; deleteByTag) and ran the local suite: bentocache (16) and memory (19) pass, the full cache unit run (90) passes, and pnpm --filter @orpc/experimental-cache type:check is clean.

Pullfrog  | Fix it ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found in the TieredCacheStore delta. Non-approving for the same reason as the prior review: the single open Pullfrog thread on the revalidate record order (middleware.ts:136) remains open.

Reviewed changes

This run reviews the delta since 6ee2f55 — the single commit 84268ab6:

  • Added TieredCacheStore (exported from the @orpc/experimental-cache barrel) — layers two or more stores front to back; a miss in any tier fills from the next via a recursive getOrSetFrom, so only the last tier runs the user's fill, and revalidate fans out to every tier through Promise.all. Each tier receives the shared options with ttl/swr capped against the tier's caps (Math.min, with an undefined option falling back to the tier cap), so a front tier's copy expires before the entry behind it can change.
  • Documented the tier semantics — the docs page gains a "Tiered Store" section (anchor #tiered-store, matching the class JSDoc @see), covering a miss filling from the next tier, revalidation reaching every tier, and the "measure ttl from when it stored its copy — cap it on front tiers" caveat. The barrel export test now asserts TieredCacheStore.
  • Tested with teeth — a shared store-contract run against two memory tiers, plus tier-specific tests: a front miss fills from a pre-populated back tier without invoking the user's fill; per-tier ttl/swr caps are honored on both capped and uncapped requests (expiry asserted under a frozen clock); waitUntil is handed out by every stale tier (two refresh promises for one two-tier stale request, with the front's refresh copying the back's stale value while the back's refresh runs the real fill); and revalidate reaches all tiers.

I traced the layered-SWR flows (stale front + stale back, stale front + fresh back, hard-miss front + stale back) and verified the recursion, the cap fallback rules, and the next returns-.output-only delegation all behave as documented and as the assertions describe; the local suite passes (tiered 11, full cache unit run 101) and pnpm --filter @orpc/experimental-cache type:check is clean. The one prior thread (middleware.ts:136) is untouched by this commit.

Pullfrog  | Fix it ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

…mplementation-09e313

# Conflicts:
#	README.md
#	apps/content/docs/api-reference.mdx
#	apps/content/package.json
#	packages/ai-sdk/README.md
#	packages/arktype/README.md
#	packages/bun/README.md
#	packages/bun/package.json
#	packages/bun/src/index.test.ts
#	packages/bun/src/index.ts
#	packages/client/README.md
#	packages/cloudflare/README.md
#	packages/cloudflare/package.json
#	packages/cloudflare/src/index.test.ts
#	packages/contract/README.md
#	packages/effect/README.md
#	packages/evlog/README.md
#	packages/hibernation/README.md
#	packages/json-schema/README.md
#	packages/msw/README.md
#	packages/nest/README.md
#	packages/next/README.md
#	packages/node/README.md
#	packages/openapi/README.md
#	packages/opentelemetry/README.md
#	packages/pinia-colada/README.md
#	packages/pino/README.md
#	packages/publisher/README.md
#	packages/ratelimit/README.md
#	packages/server/README.md
#	packages/shared/README.md
#	packages/shared/src/object.test.ts
#	packages/swr/README.md
#	packages/tanstack-query/README.md
#	packages/trpc/README.md
#	packages/valibot/README.md
#	packages/zod/README.md
#	pnpm-lock.yaml
…and coalesce fills through @orpc/experimental-lock

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The Redis rewrite moves fill coalescing into @orpc/experimental-lock and drops the per-key generation fence that b7d077f2 added, so a lock lease can expire under a slow fill and let the older fill overwrite the newer one. The new read path also deletes the entry key unconditionally after a non-atomic version check, which can discard a fresh entry.

Reviewed changes

This run reviews the PR-owned delta since the prior pullfrog review at 84268ab6 (merge 7b9bd3d4 plus 18715c68, 20976bfa, 76a79a58):

  • Replaced the Redis-family Lua scripts with single-key commands (GET/MGET/SET PX/DEL/INCR) and moved fill coalescing onto the shared Locker from @orpc/experimental-lock, so the Redis/Upstash/Bun stores work on standalone servers and Redis Cluster alike and reuse the lock adapters.
  • Added a Cacheable adapter (CacheableCacheStore, ./cacheable subpath) that backs entries onto a Cacheable instance and invalidates tags through its tag service, which the store enables on construction.
  • Dropped the BentoCache adapter and dependency — no dangling references remain in source, tests, exports, tsconfig, docs, or the lockfile.
  • Removed the generation-fence write guard (g:<key> counter, startedAt) that b7d077f2 introduced to close the Redis lock-takeover race, along with its takeover tests.

The stale-while-revalidate, tag-version snapshot, evictAt expiry, and cross-client command decoding all check out on the new base; the two findings below are specific to the removed atomicity.

ℹ️ Nitpicks

  • CacheableCacheStore (like VercelCacheStore) cannot detect a revalidation that lands while a fill is running — it is noted in the class JSDoc but not on the docs page; a sentence in the Cacheable section would set the expectation for users coming from the Redis/memory adapters.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

evictAt,
}

await this.set(this.entryPrefix + encodedKey, stringifyJSON(envelope), retention === undefined ? undefined : retention * 1000)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This write is unconditional, so a fill whose lock lease expired can overwrite the entry a later fill already stored — the exact race the per-key generation fence in b7d077f2 closed. RedisLocker does not fence its callback (packages/lock/src/adapters/base-redis.ts:105) and fillOnce even lets a timed-out waiter fill without the lock (base-key-value.ts:95), so the older fill can land last and serve stale output until its expiresAt.

Technical details
# Lock-takeover write race is no longer fenced

## Affected sites
- `packages/cache/src/adapters/base-redis.ts:144` — `set` runs unconditionally; there is no generation/token check
- `packages/cache/src/adapters/base-key-value.ts:81-100` — `fillOnce` fills without the lock after `LockTimeoutError`
- `packages/lock/src/adapters/base-redis.ts:105-110` — `lock` only guards acquisition/release; the callback keeps running after the lease expires
- Removed protection: `b7d077f2`'s `GET`/`STORE`/`RELEASE` scripts with the `g:<key>` generation and the takeover tests that asserted the later fill wins

## Required outcome
- A fill that lost its lock must not store over the entry a later fill wrote, or the store must document that the "later fill wins" guarantee is dropped. Plausible fixes: keep a generation key and compare-and-set before writing, or re-check that the entry is still absent/stale immediately before `set`.

## Open questions for the human
- Was dropping the fence a deliberate tradeoff for Redis Cluster compatibility, or did the rewrite overlook it? `b7d077f2` added it specifically to resolve Copilot thread 3969048426.

Comment thread packages/cache/src/adapters/base-redis.ts Outdated
… and eviction handling in the key-value base

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found in this delta — the millisecond conversion and the shared envelope/eviction handling read clean. Non-approving only because two prior Pullfrog threads on this PR remain open and are untouched by this commit.

Reviewed changes

This run reviews the single commit 67ea0127 since the prior pullfrog review at 76a79a5:

  • Durations switched from seconds to milliseconds — CacheGetOrSetOptions.ttl/swr, CacheEntry.expiresAt/evictAt, resolveCacheExpiry().retention, the middleware option JSDoc, the docs page, and every adapter test now use ms. resolveCacheExpiry uses Date.now() + ttl, Vercel converts back with Math.ceil(retention / 1000), and the handler plugin floors ttl/swr to whole seconds for cache-control. nowInSeconds and packages/shared/src/time.ts are gone with no dangling references.
  • Shared envelope and eviction handling moved into BaseKeyValueCacheStore — a new exported CacheEnvelope ({ output: { json, meta }, tags?, expiresAt?, evictAt? }), a decode/encode pair, and a private lock(key, fn, onTimeout) helper. getOrSet now treats undefined or isCacheEntryEvicted(entry) as a miss under the locker, and the Redis/Vercel/Cacheable reads simply return the stored entry (or undefined on a tag mismatch) instead of deleting.
  • Redis read-time delete dropped — BaseRedisCacheStore.delete and its node/Upstash/Bun del overrides are removed, so a non-atomic read-then-delete can no longer clobber a concurrent fill (the second open thread from the prior review). The node adapter also caches its client through a client() helper.
  • Memory/Cacheable/Vercel/Tiered adapters rebuilt on the shared helpers — retention is passed straight through as ms to Cacheable and via Math.ceil(ms / 1000) to Vercel; the memory store's explicit constructor is dropped.
Technical details
# Incremental review — 76a79a5 → 67ea0127

## Coverage
- Full PR diff read end-to-end via the TOC; incremental range-diff read first.
- Locally passing: memory (21), tiered (11), middleware (22), handler-plugin (13), utils (4). `cacheable`/`vercel` suites can't resolve their optional peers in this env. `apps/content` `blume build` succeeds.
- GitHub Actions all green on the head (`test_matrix` 20/22/24/26, `lint_and_typecheck`, CodSpeed, codecov, publish_preview).

## Thread disposition
- Resolved `4014175395` (non-atomic read-then-delete) — fixed by dropping the `DEL`.
- Left open `4014175382` (lock-lease takeover write race) and `3975230019` (`revalidate` record order) — unchanged by this commit, not re-raised.

Pullfrog  | Fix it ➔ | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

…mplementation-09e313

# Conflicts:
#	packages/bun/package.json
#	packages/cloudflare/package.json
#	packages/shared/src/http.test.ts
#	packages/shared/src/object.test.ts
#	pnpm-lock.yaml
- bump @standard-server/core to ~0.10.1
- import safeDecodeURIComponent from @standard-server/shared in http tests
- list cache in lock README and Cacheable in API reference
- CacheLinkPlugin reads orpc-cache-tag and orpc-cache-tag-invalidation per call, even when batched or deduplicated
- experimental_CacheRevalidationUtilsPlugin tracks query tags and invalidates affected queries when a mutation revalidates tags, resolving the mutation only after active ones refetch
- SSR friendly: the query client comes from each call, and hydrated data counts as affected until a fetch reveals its tags

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The new TanStack revalidation plugin can silently drop a revalidation: toPageKey collides distinct infinite-query page params that hashKey serializes to the same string without throwing. A scope issue in the untagged-query fallback is noted inline as well. The rest of the delta — CacheLinkPlugin's per-call header reading under batching/dedupe, and the query/infinite tracking and invalidateQueries await semantics — checks out.

Reviewed changes

This run reviews the single commit 837bd285 since the prior pullfrog review at 67ea0127:

  • Added CacheLinkPlugin (packages/cache/src/link-plugin.ts) — a client StandardLinkPlugin that reads orpc-cache-tag / orpc-cache-tag-invalidation into a per-call context; its before: ['~batch', '~dedupe'] makes it outermost, so each caller reads its own subresponse headers.
  • Added experimental_CacheRevalidationUtilsPlugin (packages/tanstack-query/src/cache-revalidation-plugin.ts) — records per-query cache tags in module-level WeakMaps and, when a mutation response revalidates tags, invalidates the affected queries and awaits active refetches before the mutation resolves; hydrated data counts as affected until a fetch reveals its tags.
  • Wired docs and packages — Link Plugin and Cache Revalidation Plugin doc sections, barrel exports, and a new @orpc/tanstack-query → @orpc/experimental-cache dependency (package.json, tsconfig reference, lockfile), with 23 new tests.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

Comment thread packages/tanstack-query/src/cache-revalidation-plugin.ts Outdated
Comment on lines +156 to +158
if (tags === undefined) {
return query.state.data !== undefined
&& (partialMatchKey(query.queryKey, queryFilterKey) || partialMatchKey(query.queryKey, infiniteFilterKey))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When no prefix is set, the untagged-query fallback matches every query in the shared QueryClient, not just this utils' procedures. init runs once at the root (path = []), so generateOperationKey([], { type: 'query' }) is [[], { type: 'query' }], and partialMatchKey derives no path constraint from the empty inner array — only the type check remains. A mutation on one prefix-less utils tree therefore refetches another prefix-less utils tree's hydrated queries (and any non-oRPC query whose key carries a matching { type: 'query' }). The direction is safe (extra refetches, not stale serves), but the docs never warn that a prefix is what scopes this fallback.

Technical details
# Untagged-query fallback is not scoped to this utils tree without a prefix

## Affected sites
- `packages/tanstack-query/src/cache-revalidation-plugin.ts:106-108` — `queryFilterKey`/`infiniteFilterKey` are built from the root `path` (`[]`).
- `packages/tanstack-query/src/cache-revalidation-plugin.ts:156-158` — the fallback predicate.

## Required outcome
- With no `prefix`, an untagged query belonging to a different prefix-less utils tree (or an unrelated query with a matching key shape) should not be invalidated.

## Open questions for the human (optional)
- Is a `prefix` meant to be required for the revalidation plugin to be correctly scoped? If so, the plugin JSDoc and the SSR tip in `apps/content/docs/integrations/tanstack-query.mdx` should say so; otherwise the fallback needs a per-utils discriminator, which `init` at the root path cannot derive from the query key alone.

…tation

A query loading for the first time alongside a revalidating mutation, such as one batched with it, could read data before the write and keep it after the mutation succeeded. The mutation now lets such in-flight first loads land before invalidating, so they are matched by their tags and refetched.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found in this delta — the batched-first-load fix reads correct. Non-approving only because the two prior Pullfrog threads on this plugin (toPageKey page-param collisions and the prefix-less fallback scope) remain open and are untouched by this commit.

Reviewed changes

This run reviews the single commit 6bc6cfb7 since the prior pullfrog review at 837bd285:

  • Refetches first loads that raced a revalidating mutation — the mutation interceptor now awaits client.refetchQueries({ fetchStatus: 'fetching', predicate: query => query.state.data === undefined && isUtilsQuery(query) }, { cancelRefetch: false }) before invalidateQueries, so an in-flight first load — such as one batched with the mutation — lands and records its tags first; otherwise the invalidation would only join that fetch and never re-run it. With cancelRefetch: false, query.fetch returns the in-flight retryer promise, so this is a pure wait; refetchQueries defaults throwOnError to falsy and swallows fetch errors, so a failed first load still does not fail the mutation.
  • Factored the invalidate predicate into a shared isUtilsQuery closure — behavior-identical for the existing fallback path.
  • Added regression coverage — a unit test (lets a first load in flight land before revalidating it) and an e2e test (refetches a query whose first load was batched with the mutation), plus a setup() refactor of the e2e file. Both fail without the wait — the first load answers with the pre-mutation value otherwise — and both pass locally (unit 17, e2e 2).

Pullfrog  | Fix it ➔ | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

…like apart

Page tags are now stored under the page param itself, and a miss looks for the earlier equal param structural sharing would have kept, so distinct Map, Set, or class-instance cursors no longer share one slot. The SSR tip notes that a prefix scopes the fallback when several utils share a query client.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

This run reviews the single commit 23b07a1 since the prior pullfrog review at 6bc6cfb7:

  • Reworked infinite-query page tagging — toPageKey's hashKey (which collided distinct Map/Set/class-instance params) is gone: INFINITE_QUERY_CACHE_TAGS now keys page params by identity, and getPageCacheTags falls back to a replaceEqualDeep scan so an equal param kept by structural sharing still matches. A lookup that finds neither short-circuits to undefined, i.e. the conservative "counts as affected" path. This closes the prior collision thread.
  • Added regression coverage — a unit test drives two Map page params through a page:0 revalidation and asserts the query still refetches, and the BigInt test was repointed at the value-based lookup.
  • Documented scoping — the SSR tip now states that several utils trees sharing one query client need a distinct prefix to scope the untagged-query fallback.

I verified the changed suite passes locally (18/18). Non-approving only because the prior prefix-less-fallback thread and the older base-redis/middleware threads remain open and are untouched by this delta.

Pullfrog  | Fix it ➔ | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

…eaders

The revalidation plugin reuses a cache link plugin context an outer reader already placed instead of replacing it, so several readers of one call, such as a custom interceptor or a second plugin instance, all see the response tags.
…uter clients

- tags live in the query state, tied to the data commit they describe, so they survive dehydration, hydration, and persisters built on it; infinite queries keep one entry per loaded page
- cacheRouterClientInterceptor reports tags for createRouterClient calls, such as server-side rendering, through a forwarded cache link plugin context
- data committed without its tags, such as hydrated, streamed, or set with setQueryData, counts as unknown

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found in this delta — the query-state tag storage and the router-client interceptor read clean, and the changed suites pass locally. Non-approving only because three prior Pullfrog threads on this PR remain open and are untouched by these commits.

Reviewed changes

This run reviews the delta since the prior pullfrog review at 23b07a1 — commits 8efe9ed3 and 7ecc2a46:

  • Moved TanStack cache tags into query state — experimental_CacheRevalidationUtilsPlugin now stores orpcCacheTags / orpcCachePageTags (and a orpcCacheTagsDataUpdateCount guard) in query.state instead of module-level WeakMaps, so tags travel with dehydration and anyone built on it. Any commit that bypasses the plugin's own fetch (hydrate, setQueryData, streaming hydration, reset) breaks the count guard and degrades to the conservative "unknown ⇒ refetch" path.
  • Added cacheRouterClientInterceptor — a ProcedureClientInterceptor for server-side router clients (SSR TanStack) that fills the forwarded CacheLinkPluginContext with the root procedure's tags/revalidations through the shared nextWithRootCacheActivity helper, ignoring nested call()s and leaving the context untouched when the call fails or the context is not forwarded.
  • Shared the link plugin context across readers — callWithCacheTags reuses an outer reader's context instead of replacing it, and CacheLinkPluginContext.tags/revalidatedTags became optional.
  • Docs/wiring — a new "Server-Side Clients" section on the cache page plus an updated SSR/dehydration tip, and the plugin barrel export.

Local verification: cache-revalidation-plugin.test.ts (28), tests/cache-revalidation.test.ts (3), and handler-plugin.test.ts (17) pass; tsc -b packages/cache packages/tanstack-query and eslint on the three changed source files are clean.

Pullfrog  | Fix it ➔ | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Caching

2 participants