Skip to content

[FLO-13.7] Add clean-room compatibility, CI tiers, and regression promotion #34

Open

Description

@szmyty

Parent: #13
Depends on: #28, #29, #30, #31, #32, #33
Suite roadmap: #11
Downstream audit: #12

Outcome

Turn the completed scenario families into a maintainable release gate: a clean-room provider compatibility matrix, enforceable CI tiers, deterministic coverage reporting, and a safe path for promoting production or fuzz regressions.

Scope

  • Build the compatibility matrix for every Flow plan, run, checkpoint, and provider contract version current policy promises to consume.
  • Exercise independently built or immutable provider artifacts; never import sibling workspaces.
  • Define and enforce pull-request hermetic smoke/failure coverage, scheduled broader compatibility/resource coverage, and manual/release clean-room provider coverage.
  • Record exact Flow/provider revisions, artifact digests, scenario-manifest version, toolchain, timestamps, and completeness state.
  • Make unavailable optional dependencies explicit skips or unavailable evidence, not silent passes.
  • Add deterministic sharding, time/memory/output/artifact-size budgets, and retention rules.
  • Document how a bug or fuzz finding is minimized, assigned a stable scenario ID, given provenance, reviewed, and promoted.
  • Generate a machine-readable coverage report and human summary without claiming finite fixtures prove universal support.
  • Reconcile Build cross-tool orchestration and failure fixture suites #13 and [EPIC] Finish the flow suite and prove production-ready orchestration #11 with exact commits, workflow runs, provider artifacts, residual gaps, and intentional deferrals.

Acceptance criteria

  • The compatibility matrix covers every supported version edge and distinguishes supported, unsupported, unavailable, and untested combinations.
  • At least one clean-room run uses immutable provider artifacts rather than source-tree imports.
  • PR, scheduled, and release tiers have enforced budgets and honest skip semantics.
  • Canonical fixtures are drift-checked and never rewritten by CI.
  • Regression promotion preserves provenance, minimization, redistribution safety, and expected typed behavior.
  • Coverage reports expose gaps and do not overclaim.
  • Documentation, package contents, workflows, contract checks, and repository validation pass.
  • Parent Build cross-tool orchestration and failure fixture suites #13 is ready to close only after all acceptance evidence is linked and reconciled.

Non-goals


October 21, 2026 audit reconciliation

AUDIT-003 is already covered here: two native optiflow tests currently return early and count as passed when the provider executable is absent. This audit separately downloaded and hash-verified v0.1.1, then actually ran all three native adapter tests on synthetic Linux fixtures. The unchanged all-target wrapper hit its 600-second budget after 49 lifecycle and 51/81 acceptance receipts; root cause remains unestablished and the complete Rust suite is not qualified. Keep that bounded qualification follow-up here rather than inventing a production defect or raising the budget. These are current partial results, not a complete clean-room compatibility matrix or proof of macOS/temporal/static routes. Require explicit unavailable/skip reporting and independently installed artifact execution in the proper tier.

Evidence: https://github.com/egohygiene/flow/blob/audit/flow-suite-20261021T025401Z/audits/flow-suite-20261021T025401Z.md. Complete roadmap and native issue hierarchy: #11. Original scope and historical handoffs above remain preserved.

October 21, 2026 — observed moving-stable lint baseline

Before merging the source-free audit PR #83, the existing hosted Rust stable job failed strict Clippy on unchanged source/tests (Rust/Clippy 1.99.0; examples include the new assert_is_empty lint in tests/hermetic_provider_kit.rs). Observed run/job. The Rust 1.85.0 and repository-validator jobs passed. This was an inspected pre-existing audit-PR result, not a new #81 run, and it does not establish that every failure is understood. Keep moving-toolchain qualification and the previously recorded bounded-wrapper timeout visible here. #81's focused Rust 1.85.0 receipt checks do not claim a green all-target moving-stable suite. No lint allowance or timeout increase is part of that repair.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions