[FEATURE] Add evidence-based earned autonomy policy engine

[Joncallim]

Parent Epic: #184
Execution mode: implementation
VNext programme: #333
Depends on: #186, #188, #337
Consumed by: #340, #343, #190, #191
Future scheduled-proof consumer: #356 — not a dependency of #189.

Problem Statement

Forge already records canonical outcomes and capability reliability evidence, but it does not yet have one deterministic policy engine that converts sufficiently comparable independently verified history into a narrowly scoped autonomy ceiling. The old Project-centric wording is too narrow for VNext, while a global agent/model trust score would be unsafe: it would ignore Resource scope, Operation risk, package/runtime/model changes, evidence freshness and critical failures.

Desired Outcome

Forge evaluates versioned earned-autonomy policy for a precise cohort — Principal/Workforce role + Capability/Operation + Resource scope + runtime/model/harness/package/policy lineage — and produces immutable promotion/hold/demotion/revocation decisions. The decision is a ceiling only: system security, MCP admission, #336 confinement/Grant policy, Resource rules and explicit human limits can always impose stricter authority.

Every decision is deterministic, replayable and explainable from stored evidence. The evaluator consumes generic verified failure/proof evidence contracts and can close before recurring proof scheduling (#356) exists; scheduled proofs feed the same evidence path later.

User Story

As the Forge operator,
I want reliable narrowly scoped capabilities to earn lower-friction execution while immediately losing that privilege when evidence deteriorates,
So that autonomy grows from verified behavior without turning any agent/model into a broadly trusted actor.

Requirements

A. Policy/cohort identity

A versioned autonomy policy must bind at least:

• policy id/version;
• Workforce/package/workflow/harness lineage where applicable;
• Capability and Operation id/version;
• normalized Resource scope fingerprint/classification;
• risk class;
• runtime/provider/model compatibility rules;
• current operator ceiling and absolute system maximum;
• minimum comparable sample count;
• rolling evidence window/freshness;
• verified-pass/promotion and demotion thresholds;
• critical-failure/violation overrides;
• requalification rules after material lineage/scope changes;
• expiry/review policy.

Do not key authority to agentType or model name alone.

B. Autonomy ladder

Support an initial generic policy ladder:

• L0 — plan/request only;
• L1 — execute typed Operation in confinement after explicit approval;
• L2 — bounded low-risk Operation may execute, human reviews result;
• L3 — may create a branch/draft PR after required independent verification;
• L4 — may open/update ready-for-review PR after required verification;
• L5 — reserved for specifically defined reversible Operations; never a generic auto-merge level.

Levels are policy conveniences over explicit Capabilities, not magic permission bundles. The effective Grant must still be calculated per Operation/Resource.

C. Evidence eligibility

Promotion may consume only comparable #186 evidence whose required #188 independent verification/Gate state is valid. Exclude or hold on:

• worker self-assessment/unverified success;
• missing/tampered/stale evidence;
• incompatible Resource scope;
• changed Operation definition/risk semantics;
• material package/workflow/harness/runtime/model/policy change outside configured compatibility;
• unresolved critical findings/rollbacks/security violations.

Unknown evidence does not count as a pass.

D. Deterministic evaluation

Given the same policy version + evidence snapshot, evaluation must produce the same result and stable reason codes. Persist the exact evidence IDs/cohort snapshot and thresholds used. No LLM or model confidence may decide promotion/demotion.

Required results include at least promote | hold | maintain | demote | revoke | expired/requalification_required.

E. Immediate revocation/demotion

Independent of aggregate success percentage, policy can immediately demote/revoke for:

• critical verification/security finding;
• policy/Grant/confinement violation;
• rollback attributed to the Capability;
• failed required deterministic proof/verification evidence from [FEATURE] Complete verification-goal on-demand proof execution through VNext #355-compatible contracts;
• later [FEATURE] Schedule verification-goal proof runs through generic Triggers #356-scheduled proof failures when [FEATURE] Schedule verification-goal proof runs through generic Triggers #356 exists, through the same generic proof-failure evidence contract rather than a direct [FEATURE] Add evidence-based earned autonomy policy engine #189 -> [FEATURE] Schedule verification-goal proof runs through generic Triggers #356 dependency;
• tampered/missing verification evidence;
• explicit operator revocation;
• material scope/Operation/security-contract change;
• repeated bounded remediation failure according to policy.

Revocation affects new admission immediately; queued stale work must fail its current authority check.

F. Human authority

Operator-set cap/revoke decisions always dominate automated promotion. An operator may deliberately lower a ceiling without deleting evidence. Raising beyond policy/system maxima is impossible; an explicit reviewed policy change is required.

Record actor, reason and audit evidence for every human override.

G. Effective authority calculation

Before every privileged Operation/admission, calculate the intersection/minimum of:

• immutable Forge/system security ceiling;
• [FEATURE] VNext Phase 2 — secure generic execution envelope and side-effect recovery #336 confinement/Operation/Resource policy;
• MCP/security-specific ceiling where relevant;
• Resource/credential policy;
• current Mission/parent Grant envelope;
• current autonomy decision;
• operator cap/revocation;
• current budget/deadline/state.

An autonomy decision never creates a Capability that the parent Mission/Resource does not already permit.

H. Persistence

Persist immutable evaluation decisions/evidence snapshots. A rebuildable current-state projection may point to the latest applicable decision, but history is append-only. Use database ordering/version identity rather than timestamps alone for precedence where races matter.

I. Advisory rollout

Implement evaluation in dry-run/advisory mode first. Compare proposed decisions to existing human gates without changing admission. Only enable enforcement after mutation/race/replay/security review and explicit migration state proves current decisions are valid.

J. Enforcement integration

Enforce effective autonomy at the common admission boundary before:

• confined implementation Operations;
• Git branch/PR actions from [FEATURE] VNext Phase 3 — complete Software Engineering as the first generic Workforce #337;
• persistent Mission child executions ([FEATURE] VNext Phase 6 — persistent Missions, checkpoints, leases, and bounded autonomy #340);
• Trigger-initiated autonomous work ([FEATURE] VNext Phase 7 — Trigger/Event runtime with dedupe, causality, and zero-token idle #341);
• adapter side effects ([FEATURE] VNext Phase 8 — general Resource/Capability adapter ecosystem #342/[FEATURE] VNext Phase 9 — Infrastructure Ops Workforce and persistent side-effect proof #343).

Fail closed if policy/current decision/evidence cannot be loaded when a higher autonomy level is required.

K. Future scheduled-proof compatibility

Define one generic proof-failure/evidence input contract that #355 fixtures can exercise now and #356 can emit later. #189 must not import #356 modules, wait for Trigger scheduling, or encode schedule-specific semantics. Its only concern is the verified evidence identity, comparability, severity/failure class, freshness and policy scope.

Implementation Sequence

1. Policy + cohort schemas — versioned explicit identities, no enforcement.
2. Evidence eligibility selector — [FEATURE] Add capability reliability ledger #186/[FEATURE] Add independent Verification Workforce execution #188 comparison/freshness/incompatibility matrix.
3. Pure deterministic evaluator — stable reason codes and replay fixtures.
4. Immutable decisions + current projection — concurrent evaluation/precedence tests.
5. Human cap/revoke APIs — audited lower-only authority changes.
6. Dry-run/advisory integration — compare against current gates on Phase-3 fixtures.
7. Effective-authority calculator — intersection of system/Grant/Resource/autonomy/operator/budget ceilings.
8. Admission enforcement — Phase-3 branch/PR and confined low-risk Operation fixtures first.
9. Immediate revocation pipeline — critical finding, generic deterministic proof failure (using [FEATURE] Complete verification-goal on-demand proof execution through VNext #355-compatible fixture), rollback/tamper and queued stale work; no [FEATURE] Schedule verification-goal proof runs through generic Triggers #356 dependency.
10. Requalification/expiry — package/model/harness/scope/policy changes and evidence aging.
11. Future-consumer conformance — synthetic scheduled-proof evidence proves the generic contract can later accept [FEATURE] Schedule verification-goal proof runs through generic Triggers #356 output without importing [FEATURE] Schedule verification-goal proof runs through generic Triggers #356.
12. Race/replay/migration gate — only then permit policy-controlled L2+ behavior.

Primary Code Seams To Inspect First

• [FEATURE] Add capability reliability ledger #186 capability reliability tables/selectors
• [FEATURE] Add independent Verification Workforce execution #188 verification/Gate findings
• [FEATURE] Normalize execution outcomes and stop reasons #185 canonical outcomes
• [FEATURE] Complete verification-goal on-demand proof execution through VNext #355-compatible deterministic proof outcome/evidence contract available transitively through [FEATURE] VNext Phase 3 — complete Software Engineering as the first generic Workforce #337
• [FEATURE][ARCH] VNext Phase 0 — generic runtime contracts and compatibility seam #334 Grant/Principal/Resource contracts
• [FEATURE] VNext Phase 2 — secure generic execution envelope and side-effect recovery #336 Operation/confinement admission
• [FEATURE] VNext Phase 3 — complete Software Engineering as the first generic Workforce #337 branch/PR Operations and Software Engineering release fixtures
• DB schema/migration and current approval gates

Do not import future #356 Trigger/scheduling code into the autonomy engine.

Orthogonal Checkpoints

1. Cohort correctness: Resource/Operation/runtime/model/package/harness scope mixing and false comparability.
2. Evidence integrity: missing/stale/tampered/unverified successes, rollback/failure attribution.
3. Policy determinism: replay/mutation tests, thresholds, sample/window boundaries, overflow/rounding.
4. Authority containment: autonomy tries to create wider Grant/Capability or bypass MCP/security/operator ceiling.
5. Concurrency/precedence: simultaneous promotion/revoke, stale cached projection, queued work after revocation.
6. Human override: lower cap, explicit revoke, policy update/requalification and audit lineage.
7. Enforcement: advisory-vs-live migration, fail-closed missing state, restart/replay.
8. Dependency isolation: [FEATURE] Add evidence-based earned autonomy policy engine #189 closes with [FEATURE] Schedule verification-goal proof runs through generic Triggers #356 still open; synthetic scheduled-proof evidence exercises only the generic evidence contract.

Acceptance Criteria

• [FEATURE] Add capability reliability ledger #186, [FEATURE] Add independent Verification Workforce execution #188 and [FEATURE] VNext Phase 3 — complete Software Engineering as the first generic Workforce #337 are closed before [FEATURE] Add evidence-based earned autonomy policy engine #189 starts; [FEATURE] Schedule verification-goal proof runs through generic Triggers #356 is not a dependency.
• Autonomy is scoped to Capability/Operation + Resource scope + relevant runtime/package/harness/policy lineage, never globally to an agent/model.
• Promotion cannot occur below configured comparable verified sample requirements.
• Worker self-assessment/unverified successes do not count as verified passes.
• Missing/stale/tampered/incompatible evidence causes hold/requalification, never promotion.
• A critical failure/security violation/rollback/required deterministic proof failure can demote or revoke immediately regardless of aggregate percentage.
• Generic proof-failure evidence can be produced by current [FEATURE] Complete verification-goal on-demand proof execution through VNext #355-compatible fixtures and later by [FEATURE] Schedule verification-goal proof runs through generic Triggers #356 without changing autonomy semantics.
• [FEATURE] Add evidence-based earned autonomy policy engine #189 has no production import/runtime dependency on [FEATURE] Schedule verification-goal proof runs through generic Triggers #356 and can close while [FEATURE] Schedule verification-goal proof runs through generic Triggers #356 remains open.
• Material Operation/Resource/package/harness/runtime/model/policy changes trigger configured hold/requalification.
• Operator caps/revocations override automated promotion and are auditable.
• Every decision is replayable from immutable policy/evidence inputs with stable reason codes.
• Effective authority cannot bypass stricter Forge/MCP/security/Resource/Grant/budget/operator ceilings or create new Capabilities.
• Queued stale work fails current admission after revocation/demotion.
• Dry-run/advisory mode is proven before enforcement.
• No general auto-merge/production deployment authority is introduced.
• Tests cover promotion, hold, maintain, demotion, immediate revocation, expiry, requalification, human override, races, evidence loss and future scheduled-proof compatibility.

Out of Scope

• Scheduling recurring proof runs ([FEATURE] Schedule verification-goal proof runs through generic Triggers #356).
• One global trust score.
• Promotion based on model confidence/prose.
• General auto-merge/deployment.
• Broad filesystem/system/credential autonomy.
• Replacing [FEATURE] Add independent Verification Workforce execution #188 verification, [FEATURE] VNext Phase 2 — secure generic execution envelope and side-effect recovery #336 security/confinement or human final gates.
• Full UI/reporting ([FEATURE] Add reliability, autonomy, and regression reporting #191).

Implementation Scope

Very Large / trust-critical - expected as 6-9 small PRs, with advisory mode and replayable evaluator isolated before enforcement.

Technical Notes

Do not hard-code illustrative percentages into architecture. Seed conservative versioned defaults by risk class, but treat policy values as explicit reviewed configuration whose changes trigger new decision lineage. The autonomy engine should understand evidence, not how that evidence happened to be scheduled.

[Joncallim]

VNext architecture constraint — 2026-09-03 repo audit

The evidence-based autonomy policy is still a core requirement, but the scope must be generalized before implementation.

Before restoring ready-for-agent:

• replace project + capability + scope as the product-wide identity with Principal/Mission + Capability + Resource scope + Grant/policy lineage; Project remains only the coding compatibility mapping;
• an autonomy decision may raise/lower the allowed ceiling inside existing operator/system policy, but it can never mint a capability/resource scope the parent Mission/operator did not already permit;
• package/workforce prompts and agents cannot mutate their own autonomy/budget/grant ceilings;
• promotion evidence must be pinned to comparable Workforce/workflow/prompt/policy/provider/runtime/resource versions where material;
• trusted deterministic policy evaluates promotion/demotion; verifier/model confidence is evidence only;
• [FEATURE] VNext Phase 2 — secure generic execution envelope and side-effect recovery #336's secure execution and Grant enforcement must exist before a higher autonomy level can authorize a real side effect;
• branch/PR levels must consume [FEATURE] VNext Phase 3 — complete Software Engineering as the first generic Workforce #337 typed Operations and reconciliation rather than an older direct GitHub path.

ready-for-agent was removed and needs-clarification added so the current Project-centric body is not implemented as a second autonomy truth.

[Joncallim]

Hostile autonomy-policy addendum — no evidence cherry-picking, cohort gaming, retry inflation, or stale promotion after revoke

A fresh policy/adversarial pass found several ways an otherwise deterministic autonomy engine could still be gamed through which evidence it evaluates or how cohorts are defined. Determinism after input selection is not enough; the selector and current-decision precedence must also be trusted and complete.

1. The evaluator receives a trusted complete evidence snapshot, never caller-selected evidence IDs

A package/model/worker/operator request may ask “evaluate autonomy for cohort X,” but it cannot supply the favorable set of attempts that counts.

The trusted selector derives the canonical evidence window from:

• exact policy/cohort identity;
• authoritative [FEATURE] Add capability reliability ledger #186 comparable-attempt registry;
• [FEATURE] Add independent Verification Workforce execution #188 applicable Gate/verification status;
• [FEATURE] Add Project Sentinel detection and escalation flow #190 active critical findings/security events where relevant;
• current time/evidence watermark;
• policy-defined inclusion/exclusion rules.

Evaluation snapshot records all included evidence IDs and exclusion counts/reasons. An attempt cannot disappear because the worker did not mention it.

Hostile fixture: 9 successes + 1 verified failure, caller supplies only the 9 pass IDs -> evaluator still sees all 10 canonical comparable observations.

2. Cohort keys/dimensions are policy-owned and derived from canonical facts

Do not let an actor reset bad history by continuously creating a “new cohort” with a slightly changed label/config string.

• cohort dimension schema/version is part of trusted [FEATURE] Add evidence-based earned autonomy policy engine #189 policy;
• values derive from canonical Operation/Resource/package/runtime evidence, not model/user text;
• normalize identities before hashing/fingerprinting;
• unknown/unresolved dimension -> incompatible/requalification/hold, not a fresh clean cohort;
• display labels/names are never cohort identity.

Policy explicitly defines compatibility groups (e.g. which model/runtime revisions are comparable). No automatic fuzzy similarity.

3. Scope narrowing and widening have asymmetric evidence rules

Reliability for Resource scope A must not automatically authorize broader A+B.

• widening Capability/Operation/Resource/risk scope => requalification/no inherited higher autonomy unless policy has explicit conservative compositional proof;
• narrowing scope may reuse evidence only when the trusted selector proves included observations actually apply to the narrower scope and no critical scoped failure is being hidden;
• a broad cohort's percentage cannot wash out a critical failure on the exact Resource being acted upon;
• changing Resource classification/risk upward triggers immediate hold/requalification even when object identity is unchanged.

4. Retries/rework/re-verification cannot inflate sample size

Define the statistical/logical sample unit explicitly.

At minimum distinguish:

• one logical Operation/work attempt;
• transport retries;
• remediation/rework attempts;
• repeated verification runs over the same subject;
• deliberate independent later attempts.

Transport duplicate/retry is never a new reliability success. Multiple verifier passes on the same unchanged subject cannot turn one successful work attempt into N promotion samples.

Policy may measure remediation rate separately, but the promotion denominator cannot be gamed by retrying easy success until threshold is reached.

Store canonical logical-attempt identity and collapse/count according to policy deterministically.

5. Evidence window selection is fixed before seeing outcomes

Rolling-window semantics belong to the policy (time duration and/or last N logical comparable attempts). The evaluator cannot choose the favorable window after reading pass/fail results.

• DB-authoritative time/order;
• exact boundary inclusivity documented/tested;
• old evidence aging out can lower sample sufficiency and force hold/requalification;
• evidence with source/observation time controlled externally does not manipulate window order;
• policy change creates a new decision lineage rather than retrospectively rewriting old decisions.

6. Failure attribution cannot be self-labelled away

A worker/model cannot mark its failed attempt “infrastructure issue, exclude me.”

Comparable outcome/failure class comes from typed Operation/Execution/verification/adjudication evidence.

Policy may distinguish:

• capability-caused failure;
• provider/host infrastructure failure;
• admission/policy block;
• inconclusive/unknown;
• security/hard-invariant violation.

But attribution is produced by trusted deterministic/adjudicated evidence. Unknown remains unknown/hold; it is not automatically excluded as harmless.

Security/hard-invariant violations can revoke regardless of whether ordinary performance attribution is disputed.

7. Critical active evidence is a live deny input, not merely an asynchronous future decision

There must be no window where #190 records an applicable critical finding but a queued Operation still sees yesterday's autonomy promotion because the next #189 evaluation job has not run yet.

Effective authority admission should include either:

• current applicable critical finding/security-revocation state directly as a deny ceiling; or
• a DB-atomic/current revocation projection updated with the critical evidence before further privileged admission can proceed.

The async #189 decision record can explain/record the resulting revoke, but safety cannot depend on notification/eventual evaluator latency.

Likewise explicit operator revoke is a live current deny, not “wait until next evaluator tick.”

8. Human revoke/cap precedence is sticky until an explicit human/policy transition

Automated promotion may never overwrite an explicit operator lower cap/revoke.

Model separately:

• earned evidence ceiling;
• operator cap;
• explicit operator revoke/disable state;
• system/security ceiling.

If the operator removes a cap/revoke later, that is an audited new human decision. Policy then determines whether prior earned evidence can become effective again or requires fresh requalification. Do not let accumulating passes implicitly undo human revoke.

9. Concurrent promotion/demotion/revoke uses evidence watermarks + CAS precedence

A pure evaluator may run concurrently:

• evaluator A snapshots evidence through revision 100 and proposes promote;
• critical failure 101 arrives;
• evaluator B proposes revoke;
• A finishes last.

A must not overwrite B/current revoke.

Every decision proposal binds:

• policy revision;
• cohort identity;
• evidence watermark/snapshot digest;
• current decision/projection generation expected;
• current operator/security override revisions.

Commit uses CAS/rechecks current critical/override state. A stale promote is stored as historical advisory evaluation if useful or rejected, but cannot become current applicable decision after newer adverse evidence/override.

Define deterministic precedence for simultaneous applicable decisions, with explicit revoke/security/operator deny dominating promote.

10. Policy updates invalidate applicability; they do not mutate old decisions

When policy/cohort schema/thresholds change:

• old decisions remain immutable history under old policy;
• current projection becomes requalification-required/pending evaluation under the new policy as defined;
• stricter security/maximum changes constrain effective authority immediately;
• a looser policy does not automatically grant higher authority until a fresh trusted evaluation under that version succeeds;
• no retroactive rewriting of evidence outcomes to make the new policy look historically applied.

11. Expiry/freshness fails safe

When the current earned decision/evidence expires or falls below minimum fresh sample:

• effective autonomy drops/holds at the policy-defined safe baseline;
• queued future privileged work rechecks and may be denied;
• an expired L3/L4 decision cannot remain effective because the evaluator is temporarily unavailable;
• failure to load/rebuild current autonomy state for an operation requiring elevated autonomy fails closed.

L0/L1 fallback semantics are explicit per Capability/risk; never guess a permissive default.

12. Ratios/threshold arithmetic is exact and reproducible

Use integer counts and rational/fixed-point/decimal arithmetic, not binary floating-point comparisons at policy boundaries.

Test:

• threshold exactly met/not met;
• large counts/overflow;
• empty denominator;
• excluded/inconclusive samples;
• rolling-window boundary;
• duplicate logical attempts.

No NaN/rounding behavior may accidentally promote.

13. Advisory -> enforcement cutover has a complete current-state migration gate

Before enabling live admission enforcement:

• enumerate every cohort/Capability that might currently rely on elevated behavior;
• compute fresh decisions from canonical evidence under the exact enforcement policy;
• missing/ambiguous cohorts default to safe baseline/requalification, not legacy implied authority;
• compare with current manual approval behavior and review unexpected widenings;
• persist enforcement-mode revision/audit;
• prove rollback to advisory does not mint authority—normal explicit Grants/human gates remain authoritative;
• after restart, enforcement mode/current projection is reconstructed from PostgreSQL, not cache.

14. Promotion cannot create a Capability/Grant or bypass higher-risk confirmation

Even L3/L4/L5 are labels over a scoped ceiling. At admission:

requested Operation Capability/Resource
∩ Mission parent Grant
∩ current security/resource/credential policy
∩ current earned ceiling
∩ operator cap/revoke
∩ budget/deadline/state
= effective authority

If the Mission never had service.restart, an earned L5 cannot invent it. If an Operation definition requires human approval regardless of autonomy due critical risk, autonomy does not waive that immutable rule.

15. Hostile fixtures

Add:

• caller provides only successful evidence IDs -> selector includes canonical failure;
• actor changes role/display/model label to reset cohort -> canonical cohort unchanged or requalification, never fresh trust by name;
• narrow-scope successes requested for broader Resource -> no promotion inheritance;
• same success reverified ten times -> sample count remains one logical work outcome under policy;
• transport retries each succeed -> no sample inflation;
• worker marks failure “infra” -> trusted attribution controls inclusion;
• critical finding commits after promotion snapshot but before promotion CAS -> promotion cannot become current;
• operator revoke then 100 passes -> revoke remains until explicit operator change;
• old evidence expires while evaluator unavailable -> elevated autonomy ceases per policy;
• policy loosened -> no instant unreviewed promotion;
• policy tightened -> effective maximum narrows immediately/fresh requalification;
• threshold floating-point edge -> exact deterministic result;
• enforcement enabled with unknown legacy cohort -> safe baseline, not inherited hidden authority.

The strengthened invariant is:

Autonomy is earned from the complete trusted history of one policy-defined comparable cohort. Neither the actor, retries, cohort labels, evaluator timing nor stale projections can choose a more favorable evidence set or out-race a current deny.