PER-8985 feat: Playwright drop-in baseline seeding — build attributes + exec orchestration

[Shivanshu-07]

What

CLI-side support for the @percy/playwright toHaveScreenshot drop-in's baseline flow:

• @percy/client — createBuild accepts allow-listed drop-in build sources via PERCY_BUILD_SOURCE (playwright-dropin / playwright-dropin-baseline) plus dropin-baseline-candidate / dropin-baseline-setup attributes (env or explicit options). The API keys baseline auto-approval on these.
• @percy/cli-exec — generic baseline-provider discovery: any installed SDK package can declare "@percy/cli": { "baselineProvider": "..." } (no framework code enters the CLI, same scaling model as command discovery). On an EMPTY project, percy exec uploads the committed baseline screenshots as an auto-approved build ⬆️ Bump @babel/register from 7.10.4 to 7.10.5 #1 before the user's command runs (the suite never executes for the baseline), then starts the head build. Established projects get a notice pointing at the SDK's explicit playwright:setup-baseline command — the API's no-build-persisted sentinel guarantees no stray builds. Seeding dispatches by project type: web → rendered web snapshots (root DOM + image resource, the percy upload shape); app → comparison ingest (tag + tile, no render).
• @percy/core — the server-decided build source is kept on percy.build so SDKs can key on it through the healthcheck.

Depends on

• PER-8985 feat: toHaveScreenshot drop-in — route existing Playwright assertions through Percy percy-playwright#640 (the SDK provider + command)
• percy/percy-api#6178 + companion API PR (server-side decision + auto-approval)

Testing

Client + exec suites green incl. 11 new seeding specs; E2E verified against a local percy-api and on staging (empty web/app seed → head diff flow, established-project sentinel, explicit setup command).

🤖 Generated with Claude Code

[Shivanshu-07]

Claude Code PR Review

PR: #2335 • Head: 0a38a90 • Reviewers: fallback inline checklist (deep-review agent)

Summary

CLI-side drop-in baseline seeding: generic baselineProvider package discovery, seed-build creation/upload (percy exec seeds an empty project's build #1 from committed screenshots, then waits for it to finish before the head build starts), new allow-listed build source + candidate/setup attributes in @percy/client, plus two @percy/core coverage fixes inherited from master.

Review Table

Priority	Category	Check	Status	Notes
High	Security	No hardcoded secrets or credentials	Pass
High	Security	Authentication/authorization checks present	N/A	token handled by @percy/client env
High	Security	Input validation and sanitization	Pass	sources allow-listed (DROPIN_BUILD_SOURCES); env flags strict === 'true'
High	Security	No IDOR — resource ownership validated	N/A
High	Security	No SQL injection (parameterized queries)	N/A
High	Correctness	Logic is correct, handles edge cases	Pass	2 High findings fixed in 0a38a90 lineage: discovery walk now degrades to [] on fs errors (a throw previously escaped to exec's catch and disabled Percy for the run); seeding is bounded to build #1 so a pre-candidate API can no longer accrete stray baseline builds on established projects
High	Correctness	Error handling is explicit, no swallowed exceptions	Pass	never-throw seeding posture is by design; walk failures logged at debug
High	Correctness	No race conditions or concurrency issues	Pass	seed-wait closes the head-baseline race; parallel shards skip seeding
Medium	Testing	New code has corresponding tests	Pass	walk-error, containment-escape, homedir-bound, non-first-build, falsy-entry cases added
Medium	Testing	Error paths and edge cases tested	Pass
Medium	Testing	Existing tests still pass (no regressions)	Pass	all package suites + 100% coverage gates green
Medium	Performance	No N+1 queries or unbounded data fetching	Pass	walk now stops at the nearest node_modules (was: every ancestor to /)
Medium	Performance	Long-running tasks use background jobs	N/A
Medium	Quality	Follows existing codebase patterns	Pass	walk mirrors @percy/cli findModulePackages semantics exactly
Medium	Quality	Changes are focused (single concern)	Pass	the two @percy/core fixes unblock master's own red Test workflow
Low	Quality	Meaningful names, no dead code	Pass
Low	Quality	Comments explain why, not what	Pass
Low	Quality	No unnecessary dependencies added	Pass

Findings

All review findings were fixed on this PR before this report:

• High packages/cli-exec/src/baseline.js: the node_modules walk could throw (EACCES / node_modules-as-file) out of findBaselineProvider into exec's outer catch — silently disabling Percy for the whole run. Walk now try/caught → [], matching the command-discovery pattern.
• High packages/cli-exec/src/baseline.js: against an API that predates the candidate attribute, every run would seed a stray "baseline" build into established projects. Seeding now requires build-number === 1.
• Medium waitForSeedBuild uses client.getBuild — verified working with standard write project tokens against percy-api (staging E2E), so the read-token doc caveat does not bite; kept as-is.
• Low: walk bounded at the home directory and stops at the nearest node_modules (narrower import surface); provider modules confined to their declaring package; falsy provider entries filtered instead of terminating an upload worker; containment paths wrapped in the sanctioned sanitizer idiom (semgrep).

---

Verdict: PASS

[aryanku-dev]

client.getBuild's own doc comment in @percy/client says it requires a read access token, but most percy exec users run with a plain write-only project token. If GET /builds/:id 403s here, the first poll throws, maybeSeedBaseline swallows it, and every first run warns did not finish processing in time (state: undefined) — then starts the head build immediately, racing the unapproved seed. That silently defeats the seed-before-head ordering this feature depends on.

Can we verify this loop works on staging with a standard write-only token (not a full-access one)? If it 403s, the API needs to allow build-state reads for the token that created the build.

[aryanku-dev]

If every upload fails transiently (e.g. a network blip), seeded is 0 but we still finalize — creating an empty, auto-approved build #1. The project is now "established" with an empty baseline: the next run diffs against nothing and seeding never retries (build-number is no longer 1), so a one-time hiccup permanently burns the auto-seed path.

Suggest: when seeded === 0, skip the finalize (or at least don't log "Baseline established") and point at percy playwright:setup-baseline instead. Partial seeding is fine; zero is not.

[aryanku-dev]

Once a project has builds, every subsequent percy exec with committed baselines still pays this full cost on every run, forever: the provider's discoverBaselines() filesystem walk, plus a doomed createBuild POST the API answers with the sentinel, plus two info log lines ("already has builds — skipping" / "run npx percy playwright:setup-baseline"). That's permanent log noise and an extra API round-trip per run for the lifetime of the project.

Consider gating the notice so it only shows once, or having the head build response carry an "established" hint so later runs can skip the sentinel round-trip entirely.