feat: monitor baseline trust and complete M5 verification tools

.github/workflows/ci.yml

@@ -240,5 +240,18 @@ jobs:
           # directive version (observed 2026-09-09) because GOTOOLCHAIN is local here.
           go-version: "1.26.x"
           check-latest: true
+      - name: Install local attestation verifier
+        uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
+        with:
+          cosign-release: v3.1.3
+      - name: Install pinned source analyzer in an isolated environment
+        run: |
+          python3 -m venv "$RUNNER_TEMP/guarddog-venv"
+          "$RUNNER_TEMP/guarddog-venv/bin/python" -m pip install 'guarddog==3.2.0'
       - name: go test -tags integration
+        # The attestation integration test runs real Cosign on a recorded npm
+        # bundle and a tampered signature; the installer verifies its release.
+        env:
+          TRUSTDIFF_GUARDDOG_INTEGRATION: "1"
+          TRUSTDIFF_GUARDDOG_BIN: ${{ runner.temp }}/guarddog-venv/bin/guarddog
         run: go test -tags integration ./...

.goreleaser.yaml

@@ -166,6 +166,9 @@ release:
 
 homebrew_casks:
   - name: trustdiff
+    commit_author:
+      name: Abdulvahap Öğüt
+      email: 110431024+vahapogut@users.noreply.github.com
     ids:
       - trustdiff
     # The archives carry a single executable and nothing else needs linking.
@@ -194,6 +197,9 @@ homebrew_casks:
 
 scoops:
   - name: trustdiff
+    commit_author:
+      name: Abdulvahap Öğüt
+      email: 110431024+vahapogut@users.noreply.github.com
     ids:
       - trustdiff
     description: Finds trust regressions in a project's dependency tree before they land

CHANGELOG.md

@@ -8,6 +8,13 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ### Added
 
+- `watch` evaluates the exact versions in an existing baseline once or at a bounded interval, reports changes in findings and coverage, and compares current owners even when a locked release is unchanged. Online cycles bypass stale HTTP caches; offline cycles explicitly use recorded data. JSON events use the published `trustdiff.watch/1` schema and never approve or rewrite the baseline.
+- Optional local npm SLSA verification with `--verify-npm-attestations` and Cosign 3.1.3. Exact package/version/SHA512 binding, certificate identity and normal Sigstore transparency verification are required; verification errors remain unavailable rather than falling back to deps.dev. Default builds keep six direct Go dependencies.
+- Optional `--guarddog` source-analysis supplements for suspicious, identified public registry releases. The reviewed GuardDog 3.2.0 runs with its sandbox on Linux/macOS, bounded time/output/package count, exact versions, and no shell. Partial and failed analyses are explicit and prevent exit 0; metadata findings remain unchanged. Native Windows, private mirrors and ambiguous locked sources are reported unsupported rather than scanning a substitute package.
+- `policy allow <check> <package-ref>` previews one reviewed exception with required reason and expiry. `--write` preserves unrelated policy text, creates a timestamped backup and replaces the file atomically. Broad patterns, expired dates and ambiguous edits are refused.
+
+
+
 - `cache refresh --crates-dump` builds a bounded, hash-checked plain-file index of the public crates.io database for bulk `scan` runs. Scans preload only requested crates, grouped by shard, and retain the normal API path for single-package `check`. `cache status` reports snapshot age; stale metadata falls back with a diagnostic, while a damaged selected shard or a package newer than the snapshot stays unavailable. Publication times, available publishers, yank flags, checksums and current owners are indexed; scripts, dependency rows, downloads, trusted-publisher evidence and historical ownership are not invented from missing data. Refreshes preserve the previous index on failure, and clearing shares the refresh lock. Ctrl+C and supported termination signals cancel the command with exit 3 and let refresh cleanup run. (#2)
 
 - A dependency-free pnpm 10 install hook that invokes trustdiff for resolved registry packages, including repeat and frozen-lockfile installs. It preserves alias/scope/peer identities, refuses unchecked or malformed reports, and stops on blocking findings, missing binaries, timeouts and output-limit violations. Documentation identifies local exclusions and unsupported dependency sources; real pnpm lifecycle tests run against a synthetic local registry on Linux and Windows. (#3)
@@ -16,6 +23,11 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 - A partial Turkish README covering installation and the three main workflows, linked to its exact English source revision. A dedicated CI check compares the recorded README content hash with the current English document, so translation drift is visible even after a squash merge or a shallow checkout. (#4)
 
+### Changed
+
+- TD008 asks for popularity only when it can change a name-collision result. TD012 prefetches counts only for enabled, non-allowlisted subjects; active exceptions are explicit policy skips. The default low-usage check still needs scoped npm counts, preserving its findings and outage behavior. Introduced dependencies retain their on-demand TD007 count checks.
+- Current crates.io owners are explicitly kept separate from historical release maintainers. An end-to-end dump regression verifies that current ownership never demotes a historical publisher change. Public data cannot fulfill the original M5.5 ownership-at-release proposal.
+
 ### Fixed
 
 - Update the pinned artifact downloader to v8.0.1 and CodeQL SARIF uploader to v4.38.1. The downloader's stricter artifact digest validation is retained; both workflow references and the composite action use verified release commit SHAs. (#7, #8)

README.md

@@ -486,10 +486,18 @@ release. Installing the latest release still installs the versions above.
 | Bulk Cargo metadata (issue #2) | An explicit crates.io dump refresh builds a bounded local index for `scan`; see [dump usage and data limits](docs/crates-dump.md). |
 | pnpm install gate (issue #3) | A copyable `.pnpmfile.cjs` checks resolved packages and frozen-lockfile installs; see [setup and supported pnpm versions](integrations/pnpm-hook/README.md). |
 | Translated introduction (issue #4) | [Turkish](docs/readme/tr.md) covers installation and the three main workflows, with its English source revision and a freshness check. |
+| Monitor existing dependencies | [`watch`](docs/watch.md) compares pinned baseline packages on a schedule; `--once` supports an external scheduler. Current-owner changes, new advisories and lost coverage are reported without approving a new baseline. |
+| Verify npm bundles locally | [`--verify-npm-attestations`](docs/local-attestations.md) uses separately installed Cosign 3.1.3, binds the exact npm subject and SHA512 checksum, and reports failed verification as unavailable. |
+| Avoid irrelevant count requests | Download counts are lazy for TD008 and batched only where enabled TD012 needs them. Active exceptions avoid requests; default TD012 still requires counts for scoped npm packages. |
+| Inspect suspicious package code | [`--guarddog`](docs/guarddog.md) invokes separately installed GuardDog 3.2.0 in its sandbox on Linux/macOS and reports bounded, attributed supplemental results. |
+| Review narrow exceptions | [`policy allow`](docs/policy-allow.md) previews a per-package/check exception with a reason and expiry; `--write` saves it atomically with a backup. |
 
 [docs/roadmap.md](docs/roadmap.md) maps these milestones to code and verification.
 [docs/PLAN.md](docs/PLAN.md) preserves the original estimates and later proposals;
-its proposed 0.6 work is not a claim that those features have shipped.
+its historical estimates are preserved. Five M5 items are implemented above.
+The remaining historical Cargo ownership premise is unavailable from public
+registry data: current owners cannot prove who owned a crate at an earlier
+release. Baselines and `watch` record observed changes without inventing that past.
 
 ## Principles
 

docs/PLAN.md

@@ -295,3 +295,16 @@ current owners but no complete ownership-event history. Current rows do not
 prove ownership at the time of an earlier release. The bulk metadata index
 must not invent that history; see [ADR 0005](adr/0005-crates-database-dump.md)
 and [the supported dump fields](crates-dump.md).
+
+## 18. M5 implementation follow-up (2026-09-29)
+
+The remaining work was subsequently approved. Items 5.1, 5.2, 5.3, 5.4 and 5.6
+are implemented: baseline watch, opt-in local Cosign verification, lazy counts,
+opt-in GuardDog handoff and reviewed policy exceptions. Their exact contracts,
+platform requirements and tests are linked from [roadmap.md](roadmap.md).
+
+Item 5.5 cannot reconstruct historical ownership from the public dump. The
+implementation deliberately keeps current-owner rows out of per-release
+maintainer sets, with an end-to-end regression proving that boundary. Baseline
+observations and watch compare evidence collected from observation time onward;
+they do not claim ownership at an unobserved publication time.

docs/adr/0006-watch-observations.md

@@ -0,0 +1,49 @@
+# 0006: Watch reviewed baseline observations without approving changes
+
+Status: accepted
+
+## Context
+
+A pinned version can gain an advisory or its package can change owners without a
+lockfile edit. `diff` cannot notice that event. A baseline already records exact
+package versions and observed trust signals, but updating that file would approve
+the very changes a monitor should report.
+
+## Decision
+
+`watch [path]` loads an existing, nonempty baseline and the policy once. It
+evaluates those pinned versions immediately and then sequentially, waiting the
+configured interval after each completed evaluation. The interval defaults to
+one hour and accepts one minute through 24 hours. `--once` performs one evaluation
+for an external scheduler. Neither form writes the baseline or persistent watch
+state, follows lockfile edits, or sends messages to external services.
+
+Each online cycle uses a fresh loader and bypasses the HTTP cache, so a polling
+interval is not silently extended by registry or advisory cache TTLs. Offline
+runs use existing caches and explicitly cannot observe new remote information.
+The current-owner comparison replaces TD003's usual npm per-release preference:
+watch compares current owners with the reviewed baseline even when the pinned
+version and its publication-time maintainer list have not changed. Other checks
+and the existing policy, failure thresholds and unavailable-data rules apply.
+
+The first observation is always emitted. Subsequent output contains only changes
+in check coverage, skip reasons or finding facts. Elapsed age text alone does not
+trigger an event; a cooldown ending does. A missing finding is called cleared
+only when the check completed on both observations. Coverage loss is never a
+claim that a finding was resolved. Events are sorted by package and check.
+
+Human output includes the current ordinary report. JSON is newline-delimited
+`trustdiff.watch/1` events embedding the unchanged `trustdiff.report/1` document.
+SARIF and Markdown are rejected because concatenating those document formats
+would produce an ambiguous stream. `--once` returns the report's exit code;
+continuous runs keep monitoring findings and source failures. Cancellation stops
+the loop and retains the existing exit-3 contract.
+
+## Consequences
+
+Online polling has the cost of a fresh audit, so requests retain the normal
+per-host rate limits and runs never overlap. Restarting emits a new initial
+observation. Users explicitly review and refresh a baseline outside this command.
+Baseline observations prove what the registry reported at observation time; they
+do not reconstruct crates.io ownership at a release date. An event timestamp is
+the time of evaluation, not the time an upstream incident happened.

docs/adr/0007-lazy-download-counts.md

@@ -0,0 +1,54 @@
+# ADR 0007: Fetch download counts when an enabled check needs them
+
+Date: 2026-09-29
+Status: accepted
+
+## Context
+
+The loader currently prefetches downloads for every resolved package and the
+runner reads them into every subject. Most checks do not use counts. npm supports
+at most 128 unscoped names per bulk request, but scoped names need separate
+requests. The official [download count documentation](https://github.com/npm/registry/blob/main/docs/download-counts.md)
+was rechecked on 2026-09-29. A scan dominated by scoped packages pays for these
+requests even when low-usage is disabled and typosquat-suspect needs no counts.
+
+Counts are evidence, so leaving a check enabled while quietly omitting a count
+it needs would change the security result. TD012 gives registry counts precedence
+over deps.dev LOW_USAGE, including when the numeric threshold is zero. TD008 can
+use counts to identify a non-list neighbor and to decide whether an old candidate
+has enough users to lower the finding's level. TD007 also uses counts on demand
+for a newly introduced dependency.
+
+## Decision
+
+Remove download counts from the generic prefetch and subject-loading phases.
+Offer an optional PrefetchDownloads operation on the loader, forwarded by the
+baseline wrapper. The runner selects only resolved subjects with an enabled,
+applicable, non-allowlisted low-usage check for that batch. The existing npm bulk
+partitioning, memoization, partial-result handling and failure handling remain.
+Loaders without the optional operation still work through their Downloads method.
+
+TD012 obtains its count through the loader when it runs. TD008 obtains the
+candidate count only when a non-list similarly named package needs comparison,
+or after a real candidate's age permits the established-package demotion.
+TD007 keeps its existing on-demand lookups. Checks use a private copy of the
+subject's download state so a timed-out check cannot race with a later check.
+Transport failures keep their unavailable status; an unrequested count is neither
+a fabricated zero nor an outage. Active allow entries for TD008 and TD012 produce
+an explicit policy skip before those checks request data.
+
+Always loading counts was rejected because disabled and allowed findings cannot
+use them. Disabling the default low-usage check or substituting deps.dev for an
+unrequested registry count was rejected because either changes the findings.
+Moving all requests into checks without batching was rejected because it would
+replace efficient unscoped npm batches with individual requests.
+
+## Consequences
+
+Scans that disable or allow low-usage avoid counts unless TD008 or TD007 needs
+them. Default low-usage still requires counts for every evaluated package, so this
+change does not promise fewer scoped npm requests under the default policy.
+Counts shared by checks and package versions remain memoized for the run.
+Regression tests count real httptest requests as well as fake loader calls, and
+assert findings, levels, fallback behavior, outages and allow expiry. No module,
+report schema or policy schema is added.

docs/adr/0008-guarddog-handoff.md

@@ -0,0 +1,26 @@
+# ADR 0008: Opt-in GuardDog handoff for suspicious registry releases
+
+Date: 2026-09-29
+Status: accepted
+
+## Context
+
+The M5 proposal asks trustdiff to pass releases with existing trust findings to a source analyzer. The normal run must retain its small Go dependency set, metadata-only behavior and offline guarantees. GuardDog is a separately installed Python tool; silently installing it, scanning local project paths, accepting a different package version, or treating its download errors as a clean scan would break those guarantees.
+
+The upstream CLI and JSON contract were verified against GuardDog v3.2.0, commit `3da172679cb58b1c9a780f9f5d640f855be016dc`, on 2026-09-29. Its default branch has changed since older integrations were written. This release requires a kernel sandbox for extraction and source analysis; its own documentation supports Linux/macOS scanning and Docker for Windows. Its JSON can contain `issues: 0` together with `errors`, so process success alone proves nothing about coverage.
+
+## Decision
+
+Add an explicit `--guarddog` handoff after trustdiff evaluates packages. Only registry releases with an existing warn or block finding qualify. Use a separately configured executable, require exactly the reviewed version 3.2.0, and invoke its scan command with a fixed argument vector, an exact validated version, JSON output, `--sandbox`, and the argument terminator `--`. Never invoke a shell, install the tool, install the target package, or run a package lifecycle command.
+
+Run the executable in an owned empty temporary directory so GuardDog cannot interpret a registry name as a same-named directory in the user's project. Support npm, PyPI and Cargo (`crates` in GuardDog). Refuse offline mode and native Windows before process creation. On supported POSIX hosts, put the process in its own group and terminate that group on cancellation, timeout or output overflow. Limit a client to twenty scans, bound each scan to two minutes by default, and retain at most 4 MiB of JSON and 64 KiB of diagnostics per process.
+
+Keep results as an attributed report supplement. They do not erase, weaken or promote trustdiff findings. Parse the package and exact version identity, counts, rule results and risk records; do not infer success from an exit code. Distinguish a completed scan, a partial scan with upstream errors, and an unavailable scan. Empty findings mean only that the selected GuardDog version reported none. Preserve useful partial evidence when a rule fails.
+
+## Consequences
+
+The default trustdiff run acquires no new dependency, network request or process. Users who opt in must install the reviewed GuardDog release and have a supported sandbox. Linux/macOS subprocess handling is tested with a deterministic helper rather than downloaded suspect packages; native Windows receives a clear unsupported message and can use trustdiff inside a supported Linux environment instead.
+
+GuardDog's download and metadata phases require network access and run before its analysis sandbox is applied. The adapter does not claim to cap the external tool's download bytes, memory or disk usage; the limits here cover wall time, process output and package count. Operators should run source scanning in their normal isolated analysis environment. New GuardDog releases require a contract review and tests before widening the version check.
+
+Primary sources: [CLI](https://github.com/DataDog/guarddog/blob/3da172679cb58b1c9a780f9f5d640f855be016dc/guarddog/cli.py), [JSON reporter](https://github.com/DataDog/guarddog/blob/3da172679cb58b1c9a780f9f5d640f855be016dc/guarddog/reporters/json.py), [package scanner](https://github.com/DataDog/guarddog/blob/3da172679cb58b1c9a780f9f5d640f855be016dc/guarddog/scanners/scanner.py), and [sandbox description](https://github.com/DataDog/guarddog/blob/3da172679cb58b1c9a780f9f5d640f855be016dc/README.md#sandboxed-scanning).