Skip to content

test: add credential-free tests for calls only covered by live tests - #231

Merged
ffumero2003 merged 31 commits into
mainfrom
test-issue-203-credential-free-live-calls
Oct 30, 2026
Merged

ffumero2003 merged 31 commits into
mainfrom
test-issue-203-credential-free-live-calls

Conversation

@ffumero2003

@ffumero2003 ffumero2003 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Context

Part of the test-integrity audit (#97). Several calls were exercised only by live tests, which skip without B2 credentials, so go test ./... and CI never ran them:

  • hide and reveal;
  • download authorization, including b2ContentDisposition;
  • list and delete keys;
  • cancel large file;
  • the upload error mappings in base.Action.

x/transport, whose fault helpers blazer's own tests and downstream users rely on, had no test file at all. Credential-free statement coverage was 75.3%.

Changes

Test-only, plus the change's intent/credential-free-live-calls/ folder (intent, spec, plan). No production code changes.

b2/live_only_calls_wire_test.go (recording server, #205)

Req Behavior pinned
R1 Object.Hide sends one b2_hide_file with bucketId and fileName, and a scripted error is returned
R2 Bucket.Reveal deletes the newest version when it is a hide marker, and sends no delete when it is an upload
R3 Bucket.AuthToken sends fileNamePrefix and validDurationInSeconds with no disposition, and returns the token
R4 Object.AuthURL sends b2ContentDisposition and returns a URL whose Authorization and b2ContentDisposition query values match
R5 Client.ListKeys sends maxKeyCount, maps name, ID and expiry, returns the cursor, sends it back as startApplicationKeyId, and returns io.EOF on an empty page
R6 Key.Delete sends applicationKeyId
R7 Object.Cancel sends b2_cancel_large_file for an unfinished file, and refuses an uploaded one without a request

base/live_only_calls_test.go (httptest)

Req Behavior pinned
R8 A second upload on an upload URL that another upload is still using is refused by B2 (400, "more than one upload using auth token"). That maps to AttemptNewUpload, as does a connection reset before any response. A context the caller cancelled or let expire stays Punt.
R9 A cancelled context returns context.Canceled, and a deadline returns context.DeadlineExceeded with the server observing the cancellation
R10 An untrusted certificate fails with a certificate verification error, checked with errors.As

x/transport/transport_test.go (new). One test per helper: pass-through, FailureRate(0) and FailureRate(1), MatchPathSubstring, Response with Body, Stall (returns at its duration, or as soon as the request context ends), and AfterNBytes.

All coordination uses channels and context deadlines, with no fixed sleeps.

Split out or left out

Verification

Each break was applied on its own, the test run, and the code restored. Every test passes once restored.

Req Break Failing output
R1 HideFile sends an empty file name fileName got "", want "gone"
R2 Reveal inverts the hide check no request for b2_delete_file_version
R3/R4 disposition dropped from the request, then from the URL b2ContentDisposition missing; got "", want "attachment…"
R5 ListKeys drops the cursor startApplicationKeyId missing, want "k-3"
R6 Key.Delete sends an empty ID applicationKeyId got "", want "k-1"
R7 Cancel inverts the status check pending is not an unfinished large file: start
R8 the shared-token case removed from Action Action(…more than one upload using auth token…) = 3, want AttemptNewUpload
R8 the pre-response upload fault case removed Action(b2_upload_file: 0: EOF) = 2, want AttemptNewUpload
R11 Response drops its body got "", want "retry later"
R11 Stall ignores the request context returned after 1.0005s, want before 500ms when the request context ends
R2 (upload) Reveal deletes when the newest version is an upload b2_delete_file_version calls = 1, want 0
R7 (upload) Cancel allows an uploaded file Cancel uploaded file error = <nil>, want "complete is not an unfinished large file: upload"; b2_cancel_large_file calls = 1, want 0
R9 (cancelled) the request ignores cancellation (context.WithoutCancel) cancelled request did not return: context deadline exceeded
R9 (deadline) the request ignores the deadline (context.WithoutCancel) deadline request did not return: context deadline exceeded
R10 the test transport skips certificate verification authorization accepted an unknown certificate authority
R11 (one-shot) the AfterNBytes callback re-arms on each request effect calls after second upload and both responses = 2, want 1
R11 (response bytes) response bytes are not counted effect calls after short body and response = 0, want 1
R2 (double delete) Reveal deletes the hide marker twice b2_delete_file_version calls = 2, want 1
R4 (empty disposition) AuthURL drops Authorization when the disposition is empty empty disposition Authorization = "", want "download-token"
  • Credential-free coverage: 75.3% → 78.5%.
  • go test -race -count=5 -shuffle=on ./b2 ./base ./x/transport passes with B2_APPLICATION_KEY_ID, B2_APPLICATION_KEY, B2_ACCOUNT_ID and B2_ACCOUNT_KEY unset.
  • make ci passes.
  • SDK panel review (deep, all five roles): FIX in round 1. The Stall test passed even when Stall ignored the context. That is fixed (the Stall row above) and re-verified. Its nit, that the spec claimed capabilities coverage, is corrected.

Base and merge order

This PR is based on main. It originally stacked on #191's shared recording server, which merged as #205, so there is no remaining dependency and it can merge on its own.

Trigger race evidence (#176)

A temporary diagnostic (removed after recording) created a client with WithFailures(nil, Trigger(ctx), Response(503)), cancelled ctx, and sent 100 requests. go test -race -count=20 reported:

WARNING: DATA RACE
Write by x/transport.Trigger.func1.1()      x/transport/transport.go:159
Previous read by x/transport.(*options).RoundTrip()  x/transport/transport.go:70

#176 fixes it (PR #251), so this PR adds no test that depends on Trigger(ctx).

Notes for reviewers

Closes #203

…e tests

Several tests pass while blazer sends the wrong request to B2, because each
wire test builds its own fake server and most accept any body. Capture the
problem and the outcome for a shared test helper before writing a spec.

Refs #191
Spec R1-R12 define the internal/b2wiretest helper: recording, scripted
responses, upload URLs and assertions, with no new dependencies and no
public API. The plan orders the work, names a test per requirement and
the break each test must catch.

Refs #191
Add hermetic HTTP and transport coverage for calls previously exercised only by credentialed live tests.

ListKeys capabilities mapping is tracked separately in #220. The transport Trigger test is omitted because its existing race is tracked in #176.

Closes #203
Assert that Stall returns before its duration when the request context ends. Align R5 with the separate ListKeys capabilities work in #220.
Treat scripted status zero as 200, reject empty JSON assertion paths consistently, and mark the server closed before draining HTTP handlers so late requests cannot report through a completed test.

Document the path and header guarantees, the migrated test coverage, and the simulator contract NOT RUN rationale requested in PR #205.
serve and copyRequests both decoded a request body into a JSON map. Share one
decodeJSON helper so the two paths cannot drift; a body that is not a JSON
object still decodes to nil. The same change was reviewed on #211.
…server' into test-issue-203-credential-free-live-calls
@ffumero2003 ffumero2003 added this to the 0.10.0 milestone Oct 29, 2026
@ffumero2003 ffumero2003 added the skip-changelog No changelog entry needed; skips the changelog check label Oct 29, 2026
@ffumero2003
ffumero2003 requested a review from goanpeca as a code owner October 29, 2026 19:46
@ffumero2003 ffumero2003 self-assigned this Oct 29, 2026
Base automatically changed from test-issue-191-shared-recording-server to main October 29, 2026 21:22
…s on short bodies

Script newest-first and older versions for Reveal, and ensure AfterNBytes does not trigger when a request body ends before its threshold.

Part of #203

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The AfterNBytes tests do not verify one-shot behavior across requests on the same transport or response-byte counting, allowing regressions in both behaviors to pass unnoticed.

2 open issues
What changed in this PR

2 Important, 0 Nit, 0 Pre-existing.

Adds credential-free coverage for previously live-only B2 calls and transport fault helpers, supporting #203 and the test-integrity audit in #97. Production code and live tests remain unchanged.

Changes:

  • Adds wire tests for hide/reveal, download authorization, key listing/deletion, and large-file cancellation.
  • Tests upload error classification, cancellation, deadlines, TLS rejection, and transport helpers.
  • Records requirements, scope, and verification plans.

Checked the supplied changes against the requirements and transport implementation; tests were not run during this review. A human code owner must review the base-layer wire and authorization coverage.

File Description
x/​transport/​transport_test.go Adds credential-free tests for transport fault helpers.
intent/​credential-free-live-calls/​spec.md Defines behavior and coverage requirements.
intent/​credential-free-live-calls/​plan.md Records implementation order and mutation checks.
intent/​credential-free-live-calls/​intent.md Establishes the test-only scope and constraints.
base/​live_only_calls_test.go Covers upload faults, context errors, and untrusted certificates.
b2/​live_only_calls_wire_test.go Verifies public API requests and response mappings.

Review effort: Balanced

Comment thread x/transport/transport_test.go Resolved
Comment thread x/transport/transport_test.go Resolved

@david-santacruz-backblaze david-santacruz-backblaze left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The expiry fixture (b2/live_only_calls_wire_test.go:124) should be null, not 0, and the test locks in 1970 for "no expiry". The Hide, Cancel (:35,181) and Reveal no-op (:54) tests can pass for the wrong reason, so assert the specific error and that b2_list_file_versions was called. Please also show the Trigger failing under -race (#176) and add the missing break rows (R9, R10, the second case of R2 and R7, x/transport). Nits: plan.md:4 cites a "lean-code ladder" that doesn't exist, and intent.md is still draft.

Pin the scripted service error and refusal paths, use the documented null expiry fixture without asserting the existing epoch mapping, and cover the AfterNBytes callback across uploads and response bodies.

Record isolated red-to-green proofs for upload refusal, cancellation, deadlines, certificate verification and transport thresholds. Document the Trigger race under #176 and the deferred null-expiry mapping.

Part of #203
Move pasted test, race and make ci output out of the committed plan; keep the
requirement-to-break tables and a short pointer to the Trigger race in #176.

Part of #203
@ffumero2003

Copy link
Copy Markdown
Contributor Author

@david-santacruz-backblaze Thanks for the detailed review. Every point is addressed in fe2843e and 11be84d, and the branch is synced with main:

  • Expiry fixture: fixed. The b2_list_keys fixture now uses the documented expirationTimestamp: null. That exposed a product bug: Key.Expires() returns 1970-01-01 instead of the zero time (base/base.go:367). It's filed as fix(base): Key.Expires() returns 1970-01-01 for keys with no expiration #244, so the test asserts the key's name and ID rather than locking in 1970.
  • Hide, Reveal and Cancel passing for the wrong reason: fixed.
    • Hide asserts the scripted 400 bad_request ("cannot hide") and exactly one b2_hide_file.
    • Reveal's no-op case asserts one b2_list_file_versions (prefix gone) and no delete.
    • Cancel asserts the exact uploaded-file error and no b2_cancel_large_file.
  • Trigger under -race: reproduced. It races with -race -count=20 when a request runs after its context is cancelled (x/transport/transport.go:70 vs :159). Fix the x/transport data race and RoundTripper contract, the x/window divide-by-zero, and add tests #176 tracks the fix, so no test here depends on Trigger(ctx); the evidence is summarized in plan.md.
  • Missing break rows: added. R2 and R7 second cases, R9 (cancelled and deadline), R10, and both R11 cases are in the PR description and plan.md. Each break was applied alone and fails its test.
  • Nits: fixed. plan.md now cites the real AGENTS.md section ("Things agents get wrong here") and is trimmed to its proof tables; intent.md is "ready for review".

go test -race -count=5 -shuffle=on ./b2/... ./base/... ./x/transport/..., go vet ./... and make ci are clean with no B2 credentials set.

@david-santacruz-backblaze david-santacruz-backblaze left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fixes, the earlier review points all look addressed. Can we check for these issues before merge?

  • R2: assert exactly one b2_delete_file_version and that b2_list_file_versions was called. srv.Last would pass on a double delete.
  • R4: with an empty disposition, also assert Authorization is still present in the URL.
  • #244: cite it in spec.md, in the ListKeys test comment and in intent.md (open questions say "None").
  • R8 message: link a source for more than one upload using auth token (restic/restic#1207) in base/live_only_calls_test.go.
  • Stacking: confirm #191 has merged and state the base and merge order in the PR description.

Optional:

  • The R9 deadline test (1s) and the Stall check (elapsed < 500ms) depend on wall-clock time. I haven't seen either fail.
  • Paste the captured -race -count=20 output for the Trigger race (#176).

@ffumero2003

Copy link
Copy Markdown
Contributor Author

@david-santacruz-backblaze Thanks. All five are addressed in 690a8f5 and 05d3911, and the branch is synced with main:

Optional items: the captured Trigger race output is now in the description (excerpt, no local paths). I left the two wall-clock checks as they are; neither has flaked, and #251 (#176) reworks Stall cancellation, where that bound can be tightened if needed.

@ffumero2003
ffumero2003 merged commit 9e8d447 into main Oct 30, 2026
23 of 24 checks passed
@ffumero2003
ffumero2003 deleted the test-issue-203-credential-free-live-calls branch October 30, 2026 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog No changelog entry needed; skips the changelog check

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: credential-free tests for calls only covered by live tests

3 participants