feat: add Client.GetMany with batch upstream and batch backends

[molon]

Docs (in Chinese; README stays bilingual): docs/design.md and docs/design/ explain the read path, singleflight, write order and striped locks, batch reads, backends and cross-instance consistency, with sequence and data flow diagrams; GLOSSARY.md, eleven ADRs in docs/adr/, docs/faq.md and docs/faq_ZH.md (moved from the README, in both languages), docs/todo.md (moved from TODO.md/TODO_ZH.md), the measurements in docs/research/ with re-runnable scripts in tools/bench/, and CLAUDE.md. The singleflight registry and the striped lock live in internal/flight and internal/stripe, with their own unit tests.

This PR now also carries what was #12 (write order: upstream first, generation-guarded backfills), merged here so both are reviewed and released together. Its description follows in Write order below.

Also carries #10's README fixes (5e99f15, cherry-picked unchanged), so the README here matches the current API in full. Whichever of the two merges first, the other rebases cleanly.

Why

Callers often need dozens of keys at once (e.g. the cached verdicts for a batch of content items). Today there is only Client.Get, one key at a time:

• When dozens of keys miss, the upstream is called once per key (dozens of queries or API calls), even though the upstream itself can usually answer a batch in one go (one WHERE key IN (...), or one API call for many items).
• So callers end up writing the batch read, miss merging and backfill themselves outside the Client, and lose the Client's protections: singleflight, DoubleCheck, Not-Found caching, serve-stale. A real example: a service that asks a model to answer a batch at once had to write its own "if this key is already being fetched, wait for it" dedup on top.

New API (all additive)

// Optional, for upstreams that answer a batch in one call.
// Keys absent from the map do not exist. A non-nil error fails the whole batch,
// unless it is a *BatchError returned as is (not wrapped),
// which fails only the keys it lists. A whole-batch error fails every key
// even if it wraps an ErrKeyNotFound.
type BatchUpstream[T any] interface {
    GetMany(ctx context.Context, keys []string) (map[string]T, error)
}

// Optional, for backends.
type BatchCache[T any] interface {
    Cache[T]
    BatchUpstream[T]
    SetMany(ctx context.Context, values map[string]T) error
    DelMany(ctx context.Context, keys []string) error
}

func (c *Client[T]) GetMany(ctx context.Context, keys []string) (map[string]T, error)

// Per-key failures of a batch; errors.Is / errors.As see through to them.
type BatchError struct{ Errors map[string]error }

// Bounds the upstream requests one GetMany has in flight: per-key upstream.Get
// calls, or chunk calls of a BatchUpstream (default DefaultGetManyFetchConcurrency = 16).
func WithGetManyFetchConcurrency[T any](concurrency int) ClientOption[T]

// Splits the keys one GetMany sends to a BatchUpstream into calls of at most
// size keys, run concurrently (default 0: one call).
func WithGetManyChunkSize[T any](size int) ClientOption[T]

// RedisCacheConfig.ChunkSize / GORMCacheConfig.ChunkSize: keys per pipeline or
// statement of the batch methods (default DefaultChunkSize = 1000).

• BatchCache is implemented by RistrettoCache, SyncMap, RedisCache (pipelines of GET/SET/DEL rather than MGET, so it also works across slots on Redis Cluster) and GORMCache (WHERE key IN (...) and multi-row upserts), both split into chunks of ChunkSize keys. SetMany/DelMany are best effort on every backend: each key is tried and the failed keys are listed in a *BatchError. Backends that don't implement it (BigCache, Transform, custom ones) fall back to per-key calls.
• Client implements BatchUpstream, so with layered clients a batch goes down as one call per layer: memory → Redis/DB → data source.

Semantics

GetMany behaves like calling Get for each key, only batched:

1. Backend: one batch read. Each value read is handled by staleness (the whole result is returned at once): fresh → into the result; stale with WithServeStale → into the result, and all stale keys of the call are refreshed together in the background (deduplicated through asyncRefreshing like Get); rotten or stale without serve-stale → refetch; miss → next step.
2. Not-Found cache: misses are checked against it with the same fresh/stale/rotten rules as Get. Keys that don't exist are simply absent from the result.
3. Singleflight: each key that needs the upstream is claimed individually in the same singleflight Get uses (same sfKey, so WithFetchConcurrency slots apply per key). Keys already in flight from a Get or another GetMany are waited for. A Get and a GetMany of the same key, or two overlapping GetManys, fetch the shared key once.
4. DoubleCheck: after claiming and before going upstream, claimed keys are re-checked against backend / Not-Found cache with the same rules as Get (in the default DoubleCheckAuto, only keys whose stripe this Client wrote since the lookup) (a fresh value or a fresh cached not-found settles the key; anything else, including a failed re-check, goes upstream).
5. Upstream: the claimed keys still missing go out as one upstream.GetMany if the upstream implements BatchUpstream, split into calls of WithGetManyChunkSize keys if set, otherwise as upstream.Gets each run exactly as Get would; at most WithGetManyFetchConcurrency (default 16) requests at a time, each answering its keys as soon as it returns. As with Get, the fetch timeout starts after the double-check. A panic or runtime.Goexit fails the keys still being fetched (not those already answered).
6. Write-back: found values → backend (SetMany when supported) after deleting their Not-Found entries; not-found keys → Not-Found cache + backend delete. As with Get's doFetch, this only touches this layer, never the upstream.

Errors: the map always holds every key that succeeded. If any key failed (backend, upstream, ctx), the error is a *BatchError with the failed keys. I chose this over "one error fails everything" because a batch is usually independent items, and one bad key shouldn't throw away the rest; it also lets a lower Client report partial failures to the layer above through the same BatchUpstream contract. Corrupt entries in Redis/GORM are reported per key the same way instead of failing the whole read.

Singleflight implementation change (internal)

golang.org/x/sync/singleflight can't tell the caller synchronously whether it became the owner of a key: DoChan decides under its lock but runs fn in a goroutine. GetMany needs that answer to build its batch, and waiting for it would deadlock two batches that each own a key the other one is waiting for. So Get and GetMany now share a tiny flightGroup (claim returns (flight, isOwner); finish releases the key before publishing the result, which is the same ordering as x/sync and what the existing DoubleCheck tests rely on). Get's behavior, hooks and error messages are unchanged; all existing tests pass unmodified. (x/sync stays in go.mod for the benchmark's semaphore.)

Compatibility

Additive only: new interfaces, new methods, new error type, one new option. Get/Set/Del behavior is unchanged. RedisCache.Get/Set were refactored to share encode/decode helpers with the batch methods, with the same behavior.

Write-back and write order: GetMany's write-back goes through setManyWithoutUpstream / delManyWithoutUpstream, the batch counterparts of setWithoutUpstream / delWithoutUpstream used by doFetch; both are guarded by the write generation described under Write order below.

Tests

batch_test.go (TDD, all run under -race, stressed with -count=30):

• BatchCache contract against SyncMap, Ristretto, Redis (+prefix), GORM (+prefix); a corrupt Redis entry doesn't hide the others
• full hit / partial hit / all miss; upstream called once with only the misses (deduplicated)
• without BatchUpstream, each key is fetched exactly as Get would (own double-check, fetch timeout and write-back, published as soon as it is fetched), claimed only when a WithGetManyFetchConcurrency slot is free; a canceled GetMany starts no more keys. Backend batch read/write is used when available
• single Get and GetMany racing on the same key → one fetch; two overlapping GetManys → shared key fetched once
• Not-Found cache: written by GetMany, honored by GetMany and Get, rotten entries refetched
• serve-stale: stale values returned at once, refreshed in one background batch
• DoubleCheck resolves a claimed key without calling the upstream
• layered clients (Ristretto → GORM → batch source): one call reaches the bottom
• whole-batch error, per-key error, panic recovery, context cancellation
• final review round: GORMCache quotes its key column (key is reserved in MySQL, so Get/GetMany/Del/DelMany failed with a syntax error there, already on main); a failed not-found cache write no longer stops the single-key write-back either (Get kept a deleted value when the not-found cache was down); a batch refresh that does not finish within the fetch timeout is logged at Warn. Verified against real MySQL 8.4 (default *_ci collation), PostgreSQL 16 and Redis 7: the BatchCache contract, 2500 keys across statements, keys differing only in case, layered GetMany
• second review round: without BatchUpstream a Get never waits for another caller's GetMany queue, and a key's fetch has its own timeout (both were per batch); a failed not-found cache write no longer skips the backend write-back of the other keys; a refresh of many keys no longer logs false failures; GORMCache never serves another key's row on a key column that is not byte-exact (MySQL's default *_ci collations made Get("abc") return ABC's value, already on main): Get/GetMany only return the row stored under exactly the requested key, and upserts rewrite the key column so a shared row belongs to the key written last. README asks for a *_bin collation on MySQL; README batch examples compile
• review fixes: the per-key fallback respects WithGetManyFetchConcurrency; a whole-batch error that wraps or joins an ErrKeyNotFound fails every key and caches no not-found; a slow double-check does not eat the upstream's fetch timeout; the same whole-batch rule now also covers the backend and the not-found cache (such an error is a failure, not a batch of misses), and the per-key error keeps the original error for errors.Is/errors.As while IsErrKeyNotFound reports false for it; a double-check whose batch read fails falls through to the upstream like Get, and a Get that joins a failed batch fetch gets the failure, not a not-found

Existing tests pass unmodified with go test -short -p=1 ./... (the CI command) and without -short; golangci-lint v2.13.2: 0 issues.

Pre-existing on main: TestClientStaleHandling, TestNotFoundCacheStale and TestEntryWithClient reported data races under -race (test counters touched by background refreshes, and a refresh outliving the test's Close). A separate test-only commit fixes them (atomic counters, wait for the background refresh before reading its result or closing); no library code changes. The full suite is now clean under -race.

Backward compatibility

Existing callers only use Get/Set/Del, the backends, WithFetchConcurrency, WithDoubleCheck, Not-Found caching, serve-stale, write propagation and the test hooks. None of that changes. What I checked, line by line against main:

Single-key Get after replacing x/sync/singleflight with flightGroup

	main (x/sync)	this PR
coalescing	first caller runs fn in a new goroutine, others join	same: claim under a mutex, owner fetches in a new goroutine
WithFetchConcurrency(n)	makeSFKey picks a random slot i:key	same makeSFKey, same slot keys
panic in the upstream	recovered inside fn → panic during upstream fetch: … to every waiter, logged with key and stack	same recover, same message, same log record
runtime.Goexit in the upstream	x/sync deletes the key and publishes nothing; waiters wait for their ctx (forever without a deadline; golang/go#52557)	changed: every waiter gets upstream fetch exited without returning (runtime.Goexit)
error propagation	same error value to every waiter, not cached	same
Forget	never called	n/a
release vs publish order	key deleted, then results sent	key deleted, then done closed
caller ctx cancelled	waiter returns context cancelled during fetch for key: …; fetch keeps running on WithoutCancel(ctx)	same for the waiter; changed: the whole fetch, double-check included, runs detached from the leader's cancellation (values kept, the leader's GORM transaction dropped), since it serves every waiter
fetchTimeout	WithTimeout(WithoutCancel(ctx)) around the fetch, after the double-check	same; the double-check is also bounded by fetchTimeout
DoubleCheckAuto (default)	on iff a not-found cache is configured	changed: re-checks only when this Client wrote the key's stripe since the lookup (hot key over a 1 ms cache: 19 upstream calls vs Enabled 18 / Disabled 30; 8000 cold keys: 1.004 cache reads per Get vs Enabled 2.0)
a not-found cache write fails during write-back	the backend is left as is (a deleted value stays)	changed: the backend is still written / its old value still deleted
test hooks	beforeSingleflightStart per caller, afterSingleflightStart per fetch, afterSingleflightEnd per caller that received the result	same call sites and counts
logs	panic during upstream fetch, async refresh failed, failed to set/delete cache entry	unchanged

Equivalence tests: get_compat_test.go pins the observable single-key behavior: 20 concurrent misses → 1 upstream call; hook counts; upstream error shared and not cached (exact message); upstream panic (message, log line, key released); runtime.Goexit (key released); fetchTimeout (exact message, upstream sees the deadline); caller cancellation (exact message, fetch outlives the caller and fills the backend); WithFetchConcurrency(3) (≤ 3 fetches); DoubleCheck race window (enabled → 1 fetch, disabled → 2); Not-Found fresh/stale/rotten (exact messages, Cached/CacheState, upstream counts, Set clears it, Del records it); serve-stale (20 stale reads → 1 refresh); backend error; write propagation through layers. The file uses only the pre-GetMany API and passes unchanged on main (I ran it against a main worktree, -race -count=3) and on this branch, so it is the baseline; all pre-existing tests also pass unmodified.

Other files: RedisCache.Get/Set now share encode/decode helpers; the decode paths (string, []byte, BinaryUnmarshaler, JSON) and error wrapping are the same as before. RistrettoCache, SyncMap, GORMCache only gain methods.

Interplay that is new (only reachable by calling GetMany with a BatchUpstream): a Get that joins a fetch a GetMany started gets that batch's result, so a key the batch upstream leaves out comes back as a plain ErrKeyNotFound (not the lower layer's Cached: true one), and fetchTimeout covers the whole batch. Callers that never call GetMany cannot hit this.

GORMCache deadlocks: concurrent batch write-backs from several instances used to deadlock. Measured with 16 workers mixing SetMany/DelMany over shared keys:

	MySQL 8.4	PostgreSQL 16
rows in map order (before)	deadlocks	~half the operations deadlocked, each stalling deadlock_timeout
one lock order (sorted SetMany; on PostgreSQL DelMany locks rows ORDER BY key COLLATE "C" FOR UPDATE, since its index follows the column's en_US collation)	3 / 1280 deadlocks (gap locks), 949 ops/s	0, 1922 ops/s
one lock order + retry deadlock victims (at most 5 attempts; not inside a WithGORMTx transaction) — shipped	0, 840 ops/s	0, 1936 ops/s
serialize writes per table (table lock) — not shipped	0, 647 ops/s	0, 1045 ops/s

GORMCache on real databases: the key column is now quoted (it is reserved in MySQL; Get/Del on main failed there with a syntax error), reads only return the row stored under exactly the requested key, upserts rewrite the key column, and Migrate creates MySQL tables with utf8mb4_0900_bin (MySQL 8.0.17 or later is now required); on an existing table Migrate checks the key column's collation and returns an error with the ALTER statement unless it is utf8mb4_0900_bin. Only each database's default isolation level is supported. TestGORMCacheOnRealDatabases runs MySQL 8.4 and PostgreSQL 16 through testcontainers (skipped without Docker; GitHub Actions has it): the BatchCache contract, a batch over several statements, case-sensitive keys, Migrate rejecting an existing case-insensitive MySQL table until it is converted, and Client reads and writes. This adds testcontainers and the MySQL/PostgreSQL GORM drivers to go.mod; only tests import them.

Benchmark

BenchmarkGetManyVsGet: 100 keys, all missing, upstream costs 1ms per call whether it answers one key or many (Apple M-series, -benchtime 20x -count 3):

	upstream calls/op	time/op
GetMany	1	~1.2 ms
loop of Get	100	~115 ms

Write order (formerly #12)

Why

Two ways a cache layer can end up holding a value its upstream does not have, and keep it until it expires:

1. Set wrote this layer before the upstream. If the upstream write failed, this layer kept the new value while the layer below (Redis, a DB-backed cache) kept the old one. Every reader of this layer then saw a value that "never happened" below.
2. A read's backfill could overwrite a newer write. A Get reads old from the upstream; before it writes old back to this layer, a Set(new) completes on both layers; then the backfill lands and this layer holds old while the upstream holds new. Same for GetMany and the serve-stale background refresh.

There was also no ordering between two concurrent Sets of one key, so L1 and L2 could end up with different winners.

What changes

Set / Del order (Client.write):

• The upstream is written first (when it implements Cache[T]), this layer only after it succeeds.
• If the upstream write fails, its state is unknown (it may have been applied and only the reply lost). This layer drops its entry and any cached not-found for the key and returns the error, so the next read goes down and sees whatever the upstream really has. A failed upstream Del likewise drops this layer's entry but records no not-found (the upstream may still have the key).
• If writing this layer fails after the upstream succeeded, the entry is dropped too (best effort), so this layer is never left with the old value.
• Del stays idempotent. On success it caches a not-found and deletes the backend entry, exactly as before.
• Not-found cache timing: Set still clears the not-found entry together with the backend write, just after the upstream write instead of before it. This is the safer order: a not-found is only cleared when there is a value below to read, and on failure it is cleared anyway so a read goes down instead of trusting a stale not-found.
• Concurrent Set/Del of one key through the same Client are serialized by a striped mutex (4096 stripes, keyed by maphash), so every layer applies them in the same order.
• Waiting for the stripe (another write, or a backfill writing this layer) gives up when the caller's ctx is done, having written nothing. A write that need not wait runs even with a done ctx, as before this PR, so an invalidating Del after the request was canceled still lands in in-memory layers.
• After the upstream write (success or failure), the key's in-flight singleflight fetches are released without being interrupted: callers already waiting still get their result, but a read that starts after Set/Del returned starts a new fetch instead of joining one that read the upstream before the write.

Backfill guard:

• Each stripe is a sync.RWMutex plus a write generation (atomic.Uint64); no per-stripe allocation, so a Client costs no more than before. Set/Del take the stripe for writing (TryLock, falling back to a ctx-aware wait) and bump the generation after the upstream write.
• doFetch (single Get, and the serve-stale refresh that goes through it) and doFetchMany (GetMany and its batch refresh) take the generation before reading the upstream. To backfill they hold the stripe for reading (TryRLock), compare the generation, and fill while still holding it. So a backfill either lands before a write (which then overwrites it) or is skipped; it can never land after a Set returned. The call still returns what it fetched.
• TryRLock never blocks: a backfill whose stripe has a write in progress is just skipped (one extra miss), so reads never wait on writes, and a batch can hold many stripes without a lock order. GetMany holds all its stripes across the value write and the not-found write.
• Fixed size, no per-key state, nothing to clean up. Two keys sharing a stripe can serialize each other's writes or skip each other's backfill; both only cost latency or one extra miss, never correctness.

SetMany / DelMany on Client: not added. #11 does not give Client batch writes (only GetMany; the backends' SetMany/DelMany are used internally for write-back), so there is no batch write path to order. If one is added later it should go through the same write path per key.

Behavior change (intended) and impact on existing callers

	before	after
Set when the upstream write fails	this layer keeps the new value, error returned	this layer has no entry (and no cached not-found), error returned
when a Set becomes visible in this layer	before the upstream write	after the upstream write succeeds (one upstream write later)
Del when the upstream delete fails	this layer deleted + not-found cached	this layer deleted, no not-found cached
Del/Set when this layer's write fails	error, upstream untouched	upstream already written; this layer's entry dropped (best effort); error returned
two concurrent Sets of one key, same Client	layers could keep different winners	serialized; layers agree
a read that fetched before a concurrent Set	could overwrite the newer value in this layer	returns what it fetched, does not write it back
a Set/Del of a key whose backfill is in progress	ran concurrently	waits for that backfill's local write (this layer only); if its ctx ends first, returns the ctx error, writes nothing upstream, and drops this layer's entry once the stripe is free
a Set/Del that has to wait for another write to its stripe	ran concurrently	waits; if its ctx ends first, same as above
GORMCache on MySQL	Get/Del failed with a syntax error (key is reserved); on a *_ci key column, keys differing in case shared a row and could read each other's value	key quoted; reads return only the row of exactly the requested key; Migrate creates new tables with utf8mb4_0900_bin (MySQL 8.0.17+, checked before creating) and returns an error for an existing table whose key column uses another collation (the error includes the ALTER)
a read that starts after Set/Del returned, while an older fetch of the key is in flight	joined the older fetch and could return the pre-write value	starts a new fetch

What callers may notice:

• Code that relied on Set "succeeding locally even if the lower layer is down" now gets a cache miss for that key instead (the error was already returned before; only what is left behind changes).

• Set/Del of the same key from many goroutines no longer run in parallel against the upstream. Different keys still do (unless they share one of 4096 stripes).

• An upstream Cache[T] whose Set/Del calls back into the same Client with any key of the same stripe (not only the same key) would now block until its ctx ends (it holds the stripe lock). Nothing in this repo does that; layered Clients are fine because each layer has its own stripes.

• While Set/Del runs, this layer's readers still see the old value (after Del, the deleted one) until this layer's own write lands: the upstream changes first. Once the call returns, the window is closed.

• A failed write's cleanup of this layer runs on a detached ctx (bounded by fetchTimeout), so it still happens when the write failed because the caller's ctx was canceled.

• Single-process guarantees only: separate processes sharing a lower layer can still briefly disagree in their memory layers, as before.

• A Set/Del that gives up waiting writes nothing upstream, but its lock is still taken in the background and then drops this layer's entry like any failed write, so a cache-aside Del after the request was canceled always clears this layer (at worst a moment after it returns).

Suggest releasing this as a minor version (the repo has no tags yet; the version bump is the signal), with the table above in the release notes.

Tests (write_order_test.go, TDD, -race)

• TestSetWritesUpstreamBeforeThisLayer: while the upstream is being written, this layer still has the old value.
• TestSetUpstreamFailureDropsThisLayersEntry: upstream Set fails → neither the new nor the old value stays, exact error message; a cached not-found is dropped too.
• TestLayerWriteFailureAfterUpstreamSuccess: the upstream write succeeds but this layer's Set/Del fails → the upstream keeps the change, the error is returned, this layer's entry and cached not-found are dropped.
• TestDelUpstreamFirst: order, idempotence; upstream Del fails → this layer's entry dropped, no not-found cached.
• TestBackfillDoesNotOverwriteAConcurrentSet: a controllable source holds the read after it has read old; a cache-aside Set(new) completes; the read returns old but the backend keeps new. Runs for Get, GetMany, the serve-stale refresh, and a not-found backfill (no not-found cached over the new value).
• TestBackfillIsSerializedWithSet: the backfill passes its check and is held just before writing old; a Set(new) starts. The Set must wait for the backfill, so no reader sees old after the Set returned, and the next read sees new. Get and GetMany.
• TestFailedWriteInvalidatesWithACanceledCtx: the caller's ctx is canceled while the upstream is written; the cleanup still drops this layer's entry on a backend that refuses a done ctx.
• TestReadAfterWriteDoesNotJoinAnOlderFetch: older reads hold a fetch in every slot after reading old; after Del returns, a new Get/GetMany starts its own fetch and sees the delete. WithFetchConcurrency 1 and 4.
• TestWriteWaitingForTheStripeRespectsCtx / TestWriteWaitingForABackfillRespectsCtx: a Set stuck behind a hung write or a held backfill returns its ctx error and writes nothing; a free stripe is written even with a done ctx.
• TestDelWithADoneCtxStillInvalidates: cache-aside Del after the request ctx was canceled still invalidates an in-memory layer.
• TestGivenUpDelStillInvalidates: a canceled Del that has to wait behind another key's backfill in its stripe returns the ctx error, and the old value is dropped once the stripe is free.
• TestReadAfterDelDoesNotJoinAFetchStartedDuringIt: with double-check, a fetch claimed while this layer was being deleted is released too, so a Get after Del returned sees the delete.
• TestRandomInterleavingsNeverReadOlderThanACompletedWrite: one writer per key Sets/Dels increasing versions through L1 → L2 → a source with random latency while readers Get/GetMany; a read that starts after write n returned never sees anything older, and once quiet every layer agrees with the source. With and without a not-found cache and double-check; it catches a disabled flight release in most runs.
• TestWriteThroughBackfillDuringUpstreamWrite: in L1 → L2, a Get that read L2 while L1's Set was writing L2 does not backfill the old value (pins where the generation is bumped).
• TestInterleavedSetsKeepLayersConsistent: 20 rounds of 20 concurrent Set/Del through L1 → L2 with random write latency; L1 never holds a value L2 does not, and a read returns L2's value. (Fails consistently with the stripe lock removed.)
• All existing tests pass unchanged, including feat: add Client.GetMany with batch upstream and batch backends #11's get_compat_test.go (single-key behavior pinned to main) and TestGetCompatWritePropagation.

go test -short -race ./... and go test -p=1 ./... pass; golangci-lint v2.13.2: 0 issues. README / README_ZH: new "Write order and consistency" notes under Write Propagation.

[Copilot]

Copilot review overview

🟡 Changes recommended

Per-key Redis failures can discard successful results, panic recovery can overwrite cached successes, and large batches lack resource bounds.

Review effort: Balanced
Findings: 2 · 3

Open (5)

• Unbounded goroutines for distinct key fetches · New
• Pipeline command errors discard successful results · New
• Upstream panic overwrites cached successes · New
• GetMany exceeds database bind-variable limits · New
• SetMany inserts exceed database bind-variable limits · New

What changed in this PR

Adds batched cache retrieval while preserving existing caching, singleflight, stale-serving, and not-found behavior.

Changes:

• Adds GetMany, batch interfaces, and per-key errors.
• Implements batching across built-in backends.
• Adds documentation, concurrency handling, tests, and benchmarks.

File	Description
types.go	Defines batch interfaces.
error.go	Adds BatchError.
client.go	Replaces singleflight internals.
batch.go	Implements batched client behavior.
syncmap.go	Adds batch operations.
ristretto.go	Adds batch operations.
redis.go	Adds pipelined batch operations.
gorm.go	Adds SQL batch operations.
batch_test.go	Tests and benchmarks batch behavior.
README.md	Documents the batch API.
README_ZH.md	Adds Chinese batch documentation.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Timeout accounting, whole-batch error classification, and unbounded fallback concurrency need correction.

3 open findings

• Preserve whole-batch errors containing not-found causes · New
• Unbounded goroutines for distinct key fetches
• Create fetch timeout after the double-check · New

4 resolved since last review

• Pipeline command errors discard successful results
• SetMany inserts exceed database bind-variable limits
• GetMany exceeds database bind-variable limits
• Upstream panic overwrites cached successes

🧠 Review effort: Balanced

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

[Copilot]

🟡 Changes recommended

Whole-batch cache errors can be hidden as misses, and upstream error flattening discards documented error identities.

3 open findings

• Distinguish whole-batch errors from per-key cache misses · New
• Preserve whole-batch not-found cache errors · New
• Preserve non-not-found error identities · New

3 resolved since last review

• Preserve whole-batch errors containing not-found causes
• Unbounded goroutines for distinct key fetches
• Create fetch timeout after the double-check

Previously missed (3)

In code that hasn't changed since last review

Avoid fatal assertion before releasing test gate

get_compat_test.go:91

If this condition is not reached, require.Eventually calls FailNow in this helper goroutine, so g.release is never closed and getConcurrently blocks until the global test timeout. Use the non-fatal assertion here so the gate is still released and the regression fails promptly.

This issue also appears in the following locations of the same file:

• line 161
• line 270

Qualify summary about single upstream call

README.md:17

This summary promises one upstream call unconditionally, but when the upstream does not implement BatchUpstream, the implementation makes one bounded Get call per miss. Qualify the claim so the feature list matches the fallback behavior documented below.

Qualify unconditional single upstream call claim

README_ZH.md:17

这里无条件写成“一次上游调用”，但上游没有实现 BatchUpstream 时会按 miss 逐个调用 Get。请注明该条件，使功能摘要与后文描述的 fallback 行为一致。

🧠 Review effort: Balanced

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.