[FEATURE] VNext Phase 4 — declarative Workforce packages and Software Engineering extraction

[Joncallim]

Parent: #333
Execution mode: implementation
Depends on: #337
Absorbs remaining useful scope from: #124 and the prompt/package portions of #121
Spec references: SPEC-0010 (Workforce Package — source resolution, immutable versions, derived revisions), SPEC-0011 (Provenance & Supply Chain), SPEC-0003 (authorization), SPEC-0005 (classification), SPEC-0008 (conformance)

Problem Statement

After Phase 3, Software Engineering will still be embedded in Forge's current database seeds, prompt files and coding-oriented assumptions. Forge cannot call Workforces plug-and-play if installing a Workforce means adding trusted executable code or editing Forge Core, nor if package prompts/policies can weaken system security ceilings.

The existing agent/workforce ontology work in #124 is now partially stale: ADR 0014 owns product-wide concepts, while ADR 0007 should survive as the Software Engineering package's role taxonomy.

Desired Outcome

Forge can inspect, install, enable, disable, update and remove versioned declarative Workforce packages without modifying Core source. Software Engineering is extracted into the first official package and passes the Phase-3 release gate unchanged.

A Workforce package may describe organisation and request capabilities; it cannot execute arbitrary installation/runtime code, carry credentials, redefine protected Operations, or weaken mandatory Forge/operator policy.

User Story

As the Forge operator,
I want Workforces to be installable and updatable like versioned capability-aware packages,
So that I can add specialized teams without turning third-party package content into trusted Forge code.

Requirements

A. Package format

Define one canonical versioned package/manifest contract containing at least:

• stable package id, display name, package version, schema version;
• compatible Forge version/range;
• role/agent definitions and display metadata;
• Workflow definitions and Work Package schemas;
• prompts/reference assets with content digests;
• required/optional Capability requests;
• supported Resource types;
• provider-neutral cognitive requirements;
• default budget profiles;
• additive package gate/evaluation definitions;
• dependencies with exact pinned version/source/digest;
• package provenance/source/digest;
• migration/update metadata where needed.

Unknown security-relevant fields fail closed. Package content is data, not code.

B. Trust boundary

A declarative package must never obtain authority merely by being parsed/installed.

• No arbitrary lifecycle/install hooks.
• No package-bundled credentials/secrets.
• No package-defined executable adapter code runs inside Core.
• Package Capability requests are maxima presented to policy/operator; Grants remain run-scoped and resource-scoped.
• Package gate/policy definitions may be stricter, never weaker than mandatory system/operator ceilings.
• Protected Operation definitions/risk classifications, secret allowlists, confinement rules and mandatory Gate semantics are not normal package-editable DB data.
• Dependency authority is the union of requested dependency capabilities and must be visible before activation.

C. Package source and lock model

Support local path and pinned Git source first. Resolve a source into an immutable installed revision plus lock/provenance data. Do not execute repository code during inspection/install.

Source resolution (per SPEC-0010 R5): Branch/tag references MUST be resolved to exact immutable commit SHAs before validation/activation. If the same packageId+version is observed with a different digest, treat it as a supply-chain version conflict — reject, do not silently replace. Derived/local revisions MUST carry explicit parent provenance and MUST NOT impersonate upstream versions.

Equivalent operator operations must exist for:

• inspect;
• install;
• list/status;
• permissions/capability review;
• enable/disable;
• update;
• remove;
• view provenance/diff.

CLI/UI spelling may follow existing Forge conventions.

D. Install/update transaction

Implement this semantic flow:

read/fetch source as data
 -> validate schema and path/content bounds
 -> calculate package + asset digests
 -> resolve/pin dependencies
 -> calculate effective requested capabilities/resources
 -> show permission/provenance diff
 -> operator/policy decision
 -> persist immutable installed revision + lock
 -> activate selected revision

Update permission expansion, including transitive dependency expansion, requires a fresh authority decision. Rollback activates a prior preserved revision; it does not rewrite history.

E. Runtime pinning

A Mission/Execution pins Workforce package, Workflow, prompts, policy, schema and dependency revisions. Updating/removing a package may not mutate the meaning of an in-flight/historical run. If removal conflicts with pinned live Missions, fail or stage removal according to explicit policy.

F. Local operator customization

Editing a packaged Workforce creates an explicit derived/local revision with parent provenance and diff. It never mutates or impersonates the upstream package identity. Reset/rebase/update flows must preserve operator intent explicitly.

G. Software Engineering extraction

Create the official Software Engineering package from the Phase-3 proven configuration. Move coding-specific role/workflow/prompt/package policy out of generic Core where possible.

Core may understand generic Mission/Workflow/Agent Run/Operation/Resource/Grant/Gate contracts; it should not require special branches for Architect, Frontend, QA, Git or PR lifecycle to support another Workforce.

H. Reviewer contract

Port the useful reviewed behavior from stale branch codex/reviewer-scope-controls into the official package rather than merging that stale branch:

• merge-gate review is scoped to the approved change/criteria;
• a blocking finding is causally introduced/exposed/materially worsened by the current change;
• adjacent pre-existing findings become separate operator-triaged issues;
• rework handoffs contain only blocking findings;
• Reviewer is read-only and produces evidence, never authority;
• review inspection/model budget is bounded by policy, not an endlessly expanding repo audit.

The repository-wide orthogonal audit skill may remain broad when explicitly invoked for a repo audit; package merge-gate review should be change-scoped.

I. Prompt/default migration

Absorb only the still-valid #124/#121 work:

• runtime-neutral package defaults rather than .codex/agents as product truth;
• editable prompts as versioned package/local-derived data;
• preserve user customizations during upgrades;
• show default-vs-derived provenance;
• do not make security-critical authority semantics normal editable prompt/config records.

Implementation Sequence

1. Manifest/schema + hostile fixtures — no installer side effects.
2. Content/provenance/lock resolver — bounded local/Git source reads and dependency graph.
3. Permission/effective-capability calculator — dependency expansion, mandatory ceiling tests.
4. Immutable package revision persistence — install/enable/disable/remove/rollback state.
5. Update transaction + permission diff — no silent authority expansion.
6. Runtime package pinning — Mission/Execution evidence references immutable revisions.
7. Local-derived revision flow — prompt/workflow customization with provenance.
8. Software Engineering package extraction — roles/prompts/workflow/gates; preserve Phase-3 behavior.
9. Reviewer-scope contract — change-scoped package reviewer + tests preventing rework scope creep.
10. Core special-case deletion pass — remove only coding-specific branches made redundant by the package; prove generic Core still supports release gate.
11. Install/update/remove/rollback E2E — including running Mission pinned to old package.

Primary Code Seams To Inspect First

• current agent_configs, workforces, workforce membership/seed code
• .codex/agents / prompt seed/export paths
• Phase-3 Mission/Workflow/Work Package contracts
• Operation/Grant/Gate policy modules
• installer/CLI architecture
• package/workforce UI/API
• review prompt/handoff/skill integration

Orthogonal Checkpoints

1. Schema/provenance: malformed manifests, path traversal, hidden files, unsupported versions, content tampering.
2. Authority: direct/transitive capability expansion, package attempt to weaken Gate/security policy, credential injection.
3. Update/history: rollback, local edits, dependency changes, running Mission pinning, package removal races.
4. Extraction: repository search for coding-only Core branches; Phase-3 release gate before/after extraction.
5. Reviewer loop: malicious/overbroad findings, unrelated pre-existing defect, repeated rework and evidence authority.
6. Final package gate: clean install -> run -> update -> pinned old run -> disable/remove -> reinstall/rollback with complete provenance.

Acceptance Criteria

• Software Engineering is loadable as a versioned package and passes the [FEATURE] VNext Phase 3 — complete Software Engineering as the first generic Workforce #337 release gate unchanged.
• Package install/update/remove does not require modifying Forge Core source.
• Package source cannot execute arbitrary host code during inspect/install/runtime merely by being a package.
• Required/optional capabilities, Resources and transitive dependency requests are inspectable before activation.
• Package policies/gates cannot weaken mandatory Forge/system/operator ceilings.
• Permission expansion on update requires a new decision.
• Running/historical Missions remain pinned to exact package/workflow/prompt/policy/dependency revisions.
• Package/dependency contents needed for evidence are digest-addressed and historical revisions remain resolvable.
• Local edits create explicit derived provenance and survive normal upgrades without silent overwrite.
• Reviewer rework can contain only change-scoped blocking findings; adjacent findings require separate triage.
• Core contains no newly introduced coding-only special case solely to make extraction pass.
• Credentials and protected Operation/security definitions are never sourced from package content.
• Hostile package conformance, normal CI and Phase-3 release proof are green.

Out of Scope

• Public marketplace/registry.
• Arbitrary executable package hooks/plugins.
• Automatic trust solely because a package is signed.
• Broad connector implementation ([FEATURE] VNext Phase 8 — general Resource/Capability adapter ecosystem #342).
• Model ensembles.

Implementation Scope

Very Large - package format/install/update/history plus flagship Workforce extraction, expected as 7-11 small PRs.

Technical Notes

Signatures/provenance answer “where did this content come from?”, not “is this safe?”. Executable capability adapters remain a separate reviewed trust boundary in Core/adapter runtime.

[github-actions]

FORGE issue validation

This Feature issue now matches the required FORGE template structure.
The validator applied ready-for-agent and removed needs-clarification if it was present.

[Joncallim]

Hostile architecture addendum — Workflow/Work Package graph validity is a trusted install/runtime contract

The current Software Engineering materializer builds a fixed deterministic graph (implementation → QA → Reviewer), so it does not expose arbitrary model-authored dependency topology today. Installable Workforces change that: package-defined Workflow/Work Package dependencies become general runtime input.

Before any package revision can be activated or any Execution instantiated from it, Forge must validate the dependency graph deterministically. A malformed graph is package/schema failure, not something the scheduler/model should discover by hanging.

1. Closed graph contract

For each Workflow revision validate at minimum:

• every node has one stable package-template identity unique within the Workflow revision;
• every dependency references a node in that same Workflow revision unless an explicit supported external dependency type exists later;
• no self-edge;
• no duplicate logical edge;
• dependency type is from a closed supported enum;
• graph is acyclic for dependency types that require DAG semantics;
• every required terminal/output/gate node is reachable from valid start/root nodes;
• no orphan/unreachable package is silently ignored;
• no dangling Gate/evaluation reference;
• fan-out, fan-in, total node/edge count and dependency depth are bounded by schema/policy before activation;
• deterministic topological order/tie-break exists for equal-ready nodes and is independent of input/map iteration order.

If future Workflow semantics intentionally support loops, those loops must be represented through an explicit bounded iteration/control construct with budget/retry ceilings—not by permitting an arbitrary graph cycle.

2. Package validation and runtime instantiation both defend the invariant

• install/update validator proves the immutable Workflow revision graph once;
• activation pins the validated graph digest/revision;
• Execution instantiation re-validates/uses only that exact pinned validated revision, never mutable package data;
• local derived revision requires validation again;
• dependency update changes graph digest and cannot silently affect running Missions;
• the runtime scheduler still fails closed if persisted graph rows violate the invariant due corruption/migration/manual DB damage.

Do not rely solely on a model-generated plan saying the ordering is sensible.

3. Work Package dependency rows are Execution-local

Once a Workflow is instantiated:

• every runtime Work Package dependency connects packages owned by the same Execution unless a separately designed cross-Execution contract explicitly exists;
• Task/Mission compatibility IDs do not substitute for exact Execution ownership;
• a superseded/replan wave cannot make a new package depend on a stale historical package unless the compatibility Workflow explicitly records an immutable inherited dependency/evidence relation with valid semantics;
• dependency eligibility is determined from canonical package lifecycle/outcome/evidence, not Task status prose.

4. Hostile fixtures

Reject or fail closed on:

• A→A;
• A→B→A;
• long cycle hidden after hundreds of nodes;
• duplicate edge under reordered JSON;
• dangling dependency ID;
• dependency into another Workflow revision/Execution;
• unreachable package with powerful Capability request;
• huge fan-out/fan-in/depth/billion-laughs-style graph payload;
• Gate references to absent package/artifact schema;
• two semantically identical manifests with reordered nodes/edges producing different graph digest/schedule;
• package update introducing a cycle while existing Mission remains pinned to old valid revision.

Property-test generated graphs: valid DAGs always admit and produce stable topo order; any added back-edge forming a cycle rejects.

5. Supply-chain/permission interaction

Effective capability/permission review must include all reachable executable nodes/dependencies, not only the nominal happy-path packages. An unreachable hidden node should be rejected as malformed rather than used to smuggle an unreviewed Capability that “probably never runs.”

Transitive package dependencies remain separate from runtime Work Package edges but need the same principles: bounded graph, no unresolved cycles unless the package spec explicitly supports them, exact source/version/digest pinning, deterministic resolution.

The invariant is: a Workforce package can describe orchestration, but it cannot make Forge's scheduler ambiguous, unbounded, cyclic, or dependent on model judgement to discover whether progress is possible.

[Joncallim]

Hostile package-source addendum — acquiring a declarative Workforce must not execute Git/content hooks

“Package content is data, not code” must apply before manifest parsing too. A naive git clone/checkout can involve configurable filters, Git LFS smudge/download behavior, submodules, credential helpers, alternate protocols or filesystem link semantics. A malicious package must not gain execution/network authority merely because Forge is trying to inspect it.

1. Harden source acquisition as a separate trusted adapter

For pinned Git sources, prefer a bounded read/archive/plumbing path that obtains exact commit/tree/blob bytes without executing repository-defined behavior.

Whichever implementation is chosen, explicitly control/disable unless separately authorized:

• repository hooks;
• clean/smudge/process filters and external textconv/diff commands;
• Git LFS automatic smudge/download;
• submodule initialization/update;
• recursive repository/dependency fetches not declared by the Workforce lock model;
• arbitrary Git protocol/ext helpers;
• ambient/global repo config that can execute programs or redirect object storage;
• credential helpers unavailable to the package-source fetcher;
• local working-tree checkout into a privileged Forge/Core directory.

Git credentials, when needed for an approved private source, are brokered to the source adapter for the exact origin and never exposed to package content/model context.

2. Source URL/origin identity is policy input

• resolve branch/tag to exact immutable commit before activation as already required;
• canonicalize/validate allowed source schemes/origins;
• package manifest/content cannot redirect the resolver to a new credentialed source outside declared dependencies;
• redirects/protocol changes require source-policy approval;
• record exact origin + commit + tree/content digest provenance;
• a local-path source is explicit operator-selected local input, not something a downloaded package can request itself.

3. Filesystem/content extraction is hostile-input parsing

Before exposing package files to normal parsers:

• cap total files, total expanded bytes, individual file size, path length/depth and dependency count;
• reject absolute paths, .. traversal, NULs and invalid/ambiguous encodings;
• do not follow symlinks out of the package root. Prefer rejecting symlink/special-file entries unless a future schema explicitly needs safe link semantics;
• reject device/FIFO/socket/special filesystem entries;
• detect case-fold / Unicode-normalization path collisions so a package cannot validate one file then load another on a different host filesystem;
• archives/compressed assets are subject to expansion-ratio/decompression-bomb limits before extraction;
• manifest/reference paths resolve against the immutable package tree, never ambient filesystem search paths.

4. Parsing remains non-executable

• YAML/JSON/template/reference parsing uses safe data parsers with bounded aliases/recursion/nesting;
• no dynamic JS/Python import/eval, template code execution, shell interpolation or executable schema callback supplied by the package;
• Markdown/reference files are data; fenced code is not run during inspect/install;
• package-defined validation expressions must be a closed declarative schema/policy language or built-in evaluator, not arbitrary code.

5. Dependency resolution repeats the same boundary

Every transitive Workforce package dependency passes through the same hardened source adapter, provenance lock, bounds and permission review. A trusted top-level package cannot make an unreviewed transitive source executable by naming it.

The resolver also validates the package dependency graph for cycles/bounds/exact pins independently from the runtime Work Package DAG described in the prior addendum.

6. Hostile fixtures

Include repositories/packages containing:

• .gitattributes filter/LFS directives;
• malicious submodule URL;
• core.*/hook-like repo config expectations;
• ext::/unexpected protocol URL;
• symlink manifest -> /etc/... and symlink escape chains;
• case-collision / Unicode-normalization collision paths;
• nested archive/decompression bomb;
• millions of tiny files / huge manifest nesting;
• YAML alias bomb;
• manifest reference to ../../;
• dependency that attempts to fetch an undeclared credentialed origin.

Prove inspection/install performs no package-authored process execution and no undeclared network/credential access.

The invariant is: Forge may fetch bytes from an approved package source; those bytes do not get to decide how Forge fetches, interprets, or executes them.

[Joncallim]

Roadmap alignment addendum — 2026-09-10

#338 inherits the deterministic Workflow kernel through #337/#367. During Software Engineering package extraction, package workflows must remain declarative definitions consumed by #367, not package-specific executable orchestration or a new checkpoint/state engine.

The package may define roles, dependency DAGs, context/output requirements and additive gates within existing ceilings. Work Package/Artifact/Gate/PostgreSQL truth remains authoritative.

[Joncallim]

Architecture reconciliation — approve one immutable package closure

Reviewed 1 October 2026 against this issue, both hostile package addenda and the later #367 alignment. Keep the existing hostile acquisition/parser boundary and declarative Workflow design. #337 remains the prerequisite; a historical ready label does not make extraction ready now.

Remaining install/update contracts

1. Inspection, approval and activation must refer to identical bytes. Build the complete immutable package/dependency/asset closure before permission review. Bind the approval to its digest, requested capabilities, Resource/data-egress requirements and relevant policy revisions. Re-reading a mutable local path/tag after approval must not activate different content. If source or policy changes, recompute/review rather than silently continuing.
2. Dependencies are required-versus-optional explicitly. Show transitive effective requests for every enabled dependency/Workflow node. An optional capability/dependency may remain unavailable, but the resulting reduced functionality must be explicit and cannot trigger an automatic broader Grant later. Activating a previously disabled optional branch is a fresh applicable permission decision, not something already covered by a hidden manifest entry.
3. Capability names alone are not a complete security diff. Surface changes in Resource scope/classification, destination/context requirements, budget defaults, proof/Gate requirements, model/runtime compatibility and dependency provenance. A prompt-only update can materially alter behaviour even without a new capability name; it still creates a new revision and resets compatibility/qualification as policy requires.
4. Additive Gates must remain bounded data. Package constraints can narrow behaviour but cannot replace mandatory criteria, reinterpret a blocked/inconclusive result as pass, invoke executable evaluators, or create an unlimited cognitive review loop. Reject contradictory/unsatisfiable required contracts at validation where determinable and explain activation blocks.
5. Disable, remove, revoke and garbage collection are different. Disabling prevents new use under policy; removal cannot delete bytes needed by pinned runs or retained evidence; security revocation can block future action on an old pinned revision. Content retention follows actual dependency/live-run/evidence references, including indirect dependency assets. A rollback to an old package does not revive a revoked adapter, credential or authority ceiling.
6. Local rebase is an explicit three-way content operation. Preserve upstream base, local derived revision and incoming upstream revision. Conflicts remain visible; no automatic overwrite of operator customisation or accidental upstream identity impersonation. The merged result is another derived revision requiring validation and applicable permission review.

Additional acceptance cases

• Local source changes between inspect and enable: only reviewed immutable closure can activate.
• Branch/tag moves or dependency is substituted after review: digest/provenance mismatch blocks.
• Optional dependency activation requests broader access: no implicit grant.
• Same capability names but broader context/destination or weaker proof request: material diff is visible and mandatory ceilings remain intact.
• Remove package while a live run pins a transitive asset: retention/disable semantics preserve reconstruction.
• Roll back to a security-revoked revision: historical bytes remain inspectable, execution remains denied.
• Local customised prompt and upstream both change: conflict cannot be silently resolved by dropping local intent.

Delivery guidance

Implement immutable acquisition/lock/approval identity before convenience install UI. Reuse #367 graph validation rather than a second package scheduler, and rerun the exact #337 proof before/after Software Engineering extraction. Signatures establish origin, not safety or permission. No package was installed, activated, removed or executed; no issue state changed.