Repository navigation
Run executable controls for the verdict evidence predicate - #298
Open
astrogilda wants to merge 3 commits into
Open
astrogilda wants to merge 3 commits into
astrogilda wants to merge 3 commits into
Conversation
The draft predicate in GenAI-Security-Project#297 says that a scope lift must come from a statement the peer signed, and that a quarantine record must be bound and have an outcome after review_by. Add a test file that runs a pinned corpus of signed records against a reference reader, with accepted and refused cases for each rule, and a workflow that installs the corpus from its own dependency group and requires it. Signed-off-by: Sankalp Gilda <23521054+astrogilda@users.noreply.github.com>
The earlier pin pointed at a commit whose own CI was red for a stale intake record. The module and corpus are unchanged between the two. Signed-off-by: Sankalp Gilda <23521054+astrogilda@users.noreply.github.com>
astrogilda
requested review from
GangGreenTemperTatum,
afogel,
bar-capsule,
fewdisc,
mamicidal,
rocklambros and
sclintonowasp
as code owners
October 18, 2026 05:59
Re-pins the group to a vectors commit whose peer statement carries the verdict the peer reached, so one statement cannot lift a record that reports another result. Adds the two refusals from the review on GenAI-Security-Project#33: a reused statement and a digest that leaves out expiry_disposition. Signed-off-by: Sankalp Gilda <23521054+astrogilda@users.noreply.github.com>
acs-board-reconciler
Bot
moved this from Needs triage
to In progress
in ACS Project Tracker
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
@rocklambros, this makes the two prose rules of #297 executable, as the open #293 did for the conformance runner with test files only.
The new test file runs records signed with published test keys against the reader in probityai/agent-evidence-vectors, which applies the ten verification steps of #297. Each refused case is one change from an accepted twin: wrong peer key, runner-signed statement, wrong vector or requirement, missing artifact, edited quarantine, overdue review. The corpus comes from a
verdict-evidencedependency group pinned to one commit in uv.lock, so it changes only when the Spec workstream moves that pin.This job is separate from ACS-Core. It runs only for the draft predicate in #297, in its own workflow, and is the only thing that installs the group. The ACS-Core suite and its CI job still install no package, as ruled on #188 and #193.
Which issue does this implement
Part of #33; refs #297.
I tested this
At 6a61e54 the full suite gives 919 passed and 1 skipped. At this PR's head, 33c6875, which pins the group to vectors commit 617cf55, the new file gives 15 passed and 2 skipped. In that commit a peer statement carries the verdict the peer reached, so one statement cannot lift a record that reports another result.
Edited 2026-10-18: the last test line now names the current head, pin and result.