Ci/pins and rustls

.github/dependabot.yml

@@ -106,7 +106,7 @@ updates:
           - patch
 
   # Python conformance + crypt4gh-interop tooling (conformance/requirements*.txt).
-  # CI-only (pyshacl/rdflib/ckanext-dcat for the FDP conformance job; crypt4gh for
+  # CI-only (pyshacl/rdflib for the FDP conformance job; crypt4gh for
   # the interop oracle) and never shipped, but still fetched-and-executed on the
   # runner — so it gets security updates like every other dependency input. The
   # top-level pins are exact; regenerate the lock with a security PR (see the header),

.github/workflows/ci.yml

@@ -127,7 +127,7 @@ jobs:
       # `ring`). All defined in scripts/ci-local.sh.
       - run: ./scripts/ci-local.sh profiles
 
-  # FDP conformance — Python (pySHACL + ckanext-dcat), isolated from the Rust
+  # FDP conformance — Python (pySHACL + harvest-profile reads), isolated from the Rust
   # build above and never shipped in the binaries. Builds the crawl producer,
   # which ingests the fixture
   # corpus, crawls the live FDP harvester-style, unions the Turtle, and runs
@@ -210,7 +210,7 @@ jobs:
           "$RUNNER_TEMP/audit-venv/bin/pip-audit" --strict --no-deps -r conformance/requirements.lock
       - name: Build the crawl producer + agreement test
         run: cargo test -p gdi-node-standalone --locked --no-run --test conformance_crawl --test conformance_agreement
-      - name: pySHACL + ckanext-dcat conformance over the crawled-and-unioned FDP graph
+      - name: pySHACL + harvest-profile conformance over the crawled-and-unioned FDP graph
         env:
           GDI_FDP_PYTHON: ${{ runner.temp }}/fdp-venv/bin/python
         run: |

.github/workflows/scheduled.yml

@@ -223,24 +223,19 @@ jobs:
   # (conformance/shapes/gdi-metadata/, conformance/ga4gh-beacon-v2/,
   # conformance/ga4gh-beacon-v2-default-model/, conformance/ga4gh-vrs-1.3/) from the commit
   # pinned in its VENDORED.md and asserts it is still byte-identical — turning the
-  # "verbatim at commit X" claim from documentation into an enforced check. `check` also
-  # runs the `pins` assertions: the userportal's deployed profile code, at the tags its own
-  # ckan/Dockerfile names, and the beacon-v2 release watch (so a consumer change surfaces
-  # here as a ticket, not a silent harvest break). It is off the per-PR `ci` workflow because
-  # it needs network access and gates nothing on a PR. A red run means a vendored file was
-  # locally edited, or upstream content or a pinned token changed — re-vendor with
-  # `scripts/vendored.sh fetch` + bump VENDORED.md, re-check and re-pin in scripts/vendored.sh,
-  # or revert the edit.
+  # "verbatim at commit X" claim from documentation into an enforced check. External pins
+  # run only in the `pins` job below. It is off the per-PR `ci` workflow because it needs
+  # network access and gates nothing on a PR. Red means a vendored file was edited locally
+  # (re-vendor with `scripts/vendored.sh fetch` and bump VENDORED.md, or revert the edit) or
+  # upstream could not be reached.
   # (The authored FDP shapes under conformance/shapes/fdp/ are not vendored — see
-  # their PROVENANCE.md — so they are not byte-checked here.) GITHUB_TOKEN raises the
-  # unauthenticated rate limit for the raw fetches.
+  # their PROVENANCE.md — so they are not byte-checked here.) Both steps fetch from
+  # raw.githubusercontent, which needs no token.
   vendored-sync:
     runs-on: ubuntu-26.04
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-      - name: Verify vendored files + external upstream pins match their pins
-        env:
-          GITHUB_TOKEN: ${{ github.token }}
+      - name: Verify vendored files match their pinned commits
         run: ./scripts/vendored.sh check
       # ...and the question `check` structurally cannot answer. It fetches each set at its
       # pinned commit — an immutable SHA — so it stays green no matter how far upstream
@@ -255,8 +250,6 @@ jobs:
       # below runs `drift gdi-metadata` fatally.
       - name: Report whether upstream has moved past our pinned commits
         continue-on-error: true
-        env:
-          GITHUB_TOKEN: ${{ github.token }}
         run: ./scripts/vendored.sh drift
 
   # macOS + Windows cross-compile of the gdi-dataset-tool CLI — moved off the per-PR
@@ -498,14 +491,13 @@ jobs:
           key: gdi-corpus-${{ hashFiles('scripts/fetch-corpus.sh') }}
       - run: ./scripts/ci-local.sh corpus
 
-  # External pin freshness: the userportal's deployed profile code, the vendored gdi-metadata
-  # shapes against upstream, the beacon-v2 release and action tags, plus the distroless
-  # runtime digest in the Dockerfile. Weekly, not per-PR, because it is a verdict about the
-  # world rather than about the commit: it goes red when a consumer contract moves, which no
-  # pull request caused, and a gate that reddens for reasons the author cannot fix is a gate
-  # people learn to ignore. An unreachable upstream is a warning. A rebuilt distroless base
-  # is a note, not a failure: `image-scan` above already fails on a fixable CVE in the pinned
-  # one.
+  # External pin freshness: the vendored gdi-metadata shapes against upstream, the beacon-v2
+  # release and action tags, plus the distroless runtime digest in the Dockerfile. Weekly,
+  # not per-PR, because it is a verdict about the world rather than about the commit: it
+  # goes red when a consumer contract moves, which no pull request caused, and a gate that
+  # reddens for reasons the author cannot fix is a gate people learn to ignore. An
+  # unreachable upstream is a warning. A rebuilt distroless base is a note, not a failure:
+  # `image-scan` above already fails on a fixable CVE in the pinned one.
   pins:
     runs-on: ubuntu-26.04
     steps:

CONTRIBUTING.md

@@ -535,7 +535,7 @@ checks each pinned SHA against its comment upstream.
 | **`sbom`** | `cargo cyclonedx --all --format json` over the whole `Cargo.lock`, producing a CycloneDX SBOM artifact. |
 | **`cross-compile`** | Five x86_64-Linux legs via `cross`, on the `release-verify` profile: the `gdi-dataset-tool` CLI for gnu and static musl, the service for gnu and static musl at `--features full`, and a musl `--features s3` build-verify. The aarch64 and macOS/Windows legs run weekly. |
 | **`e2e-smoke`** | `./scripts/e2e/run.sh`: build the service image, bring up the minimal Compose stack, pack a fixture, poll `/datasets/{id}/state` to `visible`, then assert a Beacon query, an FDP crawl, and publish/unpublish. |
-| **`conformance`** | pySHACL and ckanext-dcat over the crawled-and-unioned FDP graph, plus the dual-encoding agreement test. Python only, isolated from the Rust build and never shipped. |
+| **`conformance`** | pySHACL and the harvest-profile read check (`check_ckanext.py`) over the crawled-and-unioned FDP graph, plus the dual-encoding agreement test. Python only, isolated from the Rust build and never shipped. |
 | **`crypt4gh-interop`** | Rust↔Python crypt4gh round-trip (encrypt→decrypt, reverse, key parse) against the pinned `crypt4gh==1.8.6`. Isolated and never shipped. |
 | **`secrets`** | `gitleaks` over the working tree against the repo's own `.gitleaks.toml`, so the same exemptions govern CI and a developer machine. gitleaks is fetched from its release and checksum-verified, because it is not in install-action's tool list. |
 | **`fuzz-smoke`** | Compile-gates every `cargo-fuzz` target against the current library API, on the pinned stable toolchain — it does not fuzz. Also runs the pin- and licence-drift comparisons between the isolated fuzz manifest and the root workspace, which nothing else checks. |
@@ -545,13 +545,11 @@ checks each pinned SHA against its comment upstream.
 The `pins` leg has no `ci.yml` job: it is a verdict about the world rather than about the
 commit, so it runs weekly in `scheduled.yml` as its own `pins` job (with `PINS_STRICT=1`),
 alongside `vendored-sync`.
-`scripts/vendored.sh pins` asserts that the userportal's deployed profile code (at the tags
-its own `ckan/Dockerfile` names, so a userportal release that moves a tag without moving the
-profile passes), the GA4GH beacon-v2 latest release tag and the pinned GitHub Action tags
-still resolve to what this tree pins; `vendored.sh drift gdi-metadata` compares the vendored
-shapes against upstream `main`; and `ci-local.sh` itself compares the Dockerfile's distroless
-runtime digest against upstream, through Docker. It needs network, and unauthenticated
-GitHub API calls are rate-limited per IP, so export `GITHUB_TOKEN` — sent to
+`scripts/vendored.sh pins` asserts that the GA4GH beacon-v2 latest release tag and the pinned
+GitHub Action tags still resolve to what this tree pins; `vendored.sh drift gdi-metadata`
+compares the vendored shapes against upstream `main`; and `ci-local.sh` itself compares the
+Dockerfile's distroless runtime digest against upstream, through Docker. It needs network, and
+unauthenticated GitHub API calls are rate-limited per IP, so export `GITHUB_TOKEN` — sent to
 `api.github.com` only — to lift the budget. Real drift, including a pinned path that 404s,
 warns in `pins` and fails under `PINS_STRICT=1`, which `pins-strict`, `release` and the weekly
 job set. A rebuilt base image is a note unless `BASE_IMAGE_STRICT=1`, which only `pins-strict`
@@ -612,12 +610,11 @@ inside the `release` meta-leg are marked.
   rather than per-PR because the leg range-downloads those slices first; the corpus is
   cached
   between runs. Local: `ci-local.sh corpus`.
-- **`pins`** — external pin freshness: the userportal's deployed profile code, the vendored
-  gdi-metadata shapes against upstream, the beacon-v2 release and action tags, plus the
-  distroless runtime digest in the Dockerfile (a note there). Weekly because it is a
-  verdict about the world, not about the commit. Local:
-  `PINS_STRICT=1 ci-local.sh pins` (the job sets that so drift is fatal there; a working
-  tree only warns).
+- **`pins`** — external pin freshness: the vendored gdi-metadata shapes against upstream,
+  the beacon-v2 release and action tags, plus the distroless runtime digest in the
+  Dockerfile (a note there). Weekly because it is a verdict about the world, not about the
+  commit. Local: `PINS_STRICT=1 ci-local.sh pins` (the job sets that so drift is fatal
+  there; a working tree only warns).
 - **`notify`** — opens or updates one tracking issue on a failed weekly run. No local
   equivalent.
 - **`e2e-smoke-full`** (`e2e-full.yml`) — the full Garage + OpenBao + PME crypt4gh

Cargo.toml

@@ -206,7 +206,7 @@ reqwest = { version = "=0.13.5", default-features = false, features = ["json", "
 # rustls with `custom-provider`, which disables the implicit provider so one must be
 # installed explicitly. Shared by the service's `tls` group and the tool's startup
 # crypto-provider install; this is the surface that keeps the build on `ring` alone.
-rustls = { version = "=0.23.45", default-features = false, features = ["ring", "custom-provider", "logging", "tls12"] }
+rustls = { version = "=0.23.46", default-features = false, features = ["ring", "custom-provider", "logging", "tls12"] }
 # Mozilla CA roots as full DER certificates: `webpki-root-certs`, not the trust-anchor
 # `webpki-roots`, which reqwest 0.13 no longer consumes. Merged on top of the host platform
 # trust store, via reqwest's `tls_certs_merge` and object_store's `with_root_certificate`,

THIRD-PARTY-LICENSES.md

@@ -6150,7 +6150,7 @@ Used by:
 - ring 0.17.14
 - rustix 1.1.5
 - rustls-native-certs 0.8.4
-- rustls 0.23.45
+- rustls 0.23.46
 - scopeguard 1.2.0
 - security-framework-sys 2.17.0
 - security-framework 3.7.0

conformance/README.md

@@ -104,23 +104,18 @@ into `target/conformance/union.ttl`. It then runs:
     localized paths, and both are exercised: the enriched corpus dataset is bilingual in
     both, while the minimal one stays plain so the `LocalizedText::Plain` render path is
     covered too.
-- **`check_ckanext.py <union.ttl>`** — consumer compatibility, the nice-to-have check.
+- **`check_ckanext.py <union.ttl>`** — consumer compatibility.
   The profile the userportal deploys is `fairdatapoint_dcat_ap`
   (`ckanext.fairdatapoint.profiles.FAIRDataPointDCATAPProfile`), which subclasses
   `euro_health_dcat_ap` to add tag validation, `tags_translated` sanitising and label
   resolution. It overrides no predicate read, so the parent's predicate set is the
   deployed one. It ships in the harvester extension
-  (`gdi-userportal-ckanext-fairdatapoint`), not in `ckanext-dcat`, so a harvester bump can
-  change the profile with the `ckanext-dcat` pin standing still; the profile code of both
-  forks is watched by content, at the tags the userportal deploys (see
-  [Version pins](#version-pins)).
-
-  The check prefers the real `ckanext-dcat` `RDFParser` on the parent profile, since the
-  subclass's package is not installed in this venv and only the parent's reads are under
-  test. The standalone venv has no full CKAN, so it degrades to a direct rdflib parse
-  mirroring those exact predicate reads and prints a `[best-effort]` note. In practice that
-  fallback is the live tier: `ckanext-dcat==2.4.4` installs, but importing its `RDFParser`
-  raises `ModuleNotFoundError: No module named 'ckan'`, since the venv has no CKAN.
+  (`gdi-userportal-ckanext-fairdatapoint`), not in `ckanext-dcat`.
+
+  The real `RDFParser` needs a full CKAN install, so the check mirrors its predicate reads
+  in plain rdflib. Its docstring names the fork versions the list was last checked
+  against. CI does not watch the forks, so recheck the list if a harvest of this node
+  loses a field.
 
   Every field is required on every dataset, not merely somewhere in the graph. An
   existential check would let one healthy dataset vouch for all of them, which matters here
@@ -197,19 +192,9 @@ You can also run a check by hand once a `union.ttl` exists:
   the gdi-metadata model tag the node's static mapping table encodes. The vendored shapes
   under `shapes/gdi-metadata/` are that model; see `shapes/gdi-metadata/VENDORED.md` for
   the exact upstream commit they were copied from.
-- **pyshacl / rdflib / ckanext-dcat** — pinned in `requirements.txt` and resolved into the
+- **pyshacl / rdflib** — pinned in `requirements.txt` and resolved into the
   hash-locked `requirements.lock` (regenerate both together with
   `uv pip compile --universal --generate-hashes`); bump them in a reviewed change.
-  `ckanext-dcat` is pinned to `2.4.4`, the upstream base of the GDI fork the userportal
-  deploys (`gdi-userportal-ckanext-dcat`, with the harvester
-  `gdi-userportal-ckanext-fairdatapoint`, at the tags gdi-userportal-ckan-docker's
-  `ckan/Dockerfile` names). The profile the deployment's harvest sources name is
-  `fairdatapoint_dcat_ap`, supplied by the harvester as a subclass of the fork's
-  `euro_health_dcat_ap`, so the `ckanext-dcat` pin alone would not see a profile change.
-  That is why both profile files are watched below by content at the deployed tag, and the
-  fork's `pyproject.toml` version against this `2.4.4` pin. `check_ckanext.py` runs an
-  rdflib hand-mirror of the inherited `euro_health_dcat_ap` reads, because the real
-  `RDFParser` needs full CKAN and so never loads in this venv.
 - **Guards on all of the above.** Which gate reaches each one differs, and the difference
   matters, so it is stated per guard rather than claimed for the list. The offline
   integrity of these files is fully covered by `scripts/ci-local.sh all`; the two guards
@@ -220,21 +205,16 @@ You can also run a check by hand once a `union.ttl` exists:
     the dangerous direction, because weakening a vendored `sh:minCount` or deleting a
     `"required"` array from a Beacon schema makes everything downstream pass more easily.
   - `vendored.sh pins` (network, in the `pins` leg). Under `all`, both an unreachable
-    network and real drift warn and pass: drift here means the federation moved, not that
-    this tree broke, so it is reported rather than fatal. The weekly job, `release` and
-    `pins-strict` set `PINS_STRICT=1` and fail closed on drift. Two watches:
-    `USERPORTAL_PROFILE_PINS`, the deployed profile files by hash and the dcat fork's
-    upstream base against this `requirements.txt`, resolved at the tags the userportal's
-    own `ckan/Dockerfile` names, so a userportal release that moves a tag but neither fact
-    passes; and `EXTERNAL_PINS`, the latest GA4GH beacon-v2 release tag — the only watch on
-    a standard rather than on what the federation deploys. That one reads the releases API,
-    because a release is a tag and tags are in no file the repo contains: beacon-v2's own
-    CHANGELOG tops out at `2.0.0` while the repo is tagged `v2.2.0`, so a file-based watch
-    would be a dead guard. The same subcommand also runs `check_action_pins`, which is
-    about this repo rather than the federation: it re-resolves every
-    `uses: owner/repo@<sha> # <tag>` in the workflows and reports a SHA that no longer is
-    that tag. The `pins` leg also runs `vendored.sh drift gdi-metadata` (below), with the
-    same warn-or-fail split.
+    network and real drift warn and pass: drift here means upstream moved, not that this
+    tree broke, so it is reported rather than fatal. The weekly job, `release` and
+    `pins-strict` set `PINS_STRICT=1` and fail closed on drift. It watches `EXTERNAL_PINS`,
+    the latest GA4GH beacon-v2 release tag. That reads the releases API, because a release
+    is a tag and tags are in no file the repo contains: beacon-v2's own CHANGELOG tops out
+    at `2.0.0` while the repo is tagged `v2.2.0`, so a file-based watch would be a dead
+    guard. The same subcommand also runs `check_action_pins`, which is about this repo's own
+    pins: it re-resolves every `uses: owner/repo@<sha> # <tag>` in the workflows and reports
+    a SHA that no longer is that tag. The `pins` leg also runs
+    `vendored.sh drift gdi-metadata` (below), with the same warn-or-fail split.
   - `pip-audit` (in the `supply_chain` leg) — the advisory scan for the one corner
     `cargo deny` cannot see. It audits the `.lock` files, not the `.txt`: pip-audit resolves
     a requirements file, and the loose `.txt` can resolve a newer version than the lock pins
@@ -271,9 +251,9 @@ You can also run a check by hand once a `union.ttl` exists:
     when you touch a vendored input: `scripts/vendored.sh check` and
     `scripts/vendored.sh drift`, or a full `scripts/ci-local.sh release`.
 
-  One input has no automated watch: `shapes/fdp/` is hand-derived from the FDP v1.2
-  specification document, and no watch on a spec document would be meaningful. See its
-  `PROVENANCE.md`.
+  Two inputs have no automated watch: `shapes/fdp/`, hand-derived from the FDP v1.2
+  specification (see its `PROVENANCE.md`), and the userportal's harvest profile, which
+  `check_ckanext.py` mirrors by hand.
 
 ## Beacon wire-schema conformance