ci (release): Refactor release workflow and address readiness assessment topics - #371
ci (release): Refactor release workflow and address readiness assessment topics#371turbobobbytraykov wants to merge 33 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR strengthens the IgniteUI.Blazor.Lite release pipeline by splitting the release workflow into least-privilege jobs, adding supply-chain evidence generation (SBOM + attestations), introducing enforced bundle-size budgets, and publishing readiness documents (accessibility, performance, nullable plan). It also improves NuGet package provenance metadata and pins signing identities in-repo.
Changes:
- Refactors the GitHub release workflow into isolated build/sign/pack/evidence/SBOM/publish/attach jobs with digest-verified handoffs.
- Adds enforced static web asset bundle budgets plus reporting (
eng/Check-BundleBudget.ps1,eng/bundle-budgets.json) and publishes related docs. - Improves NuGet provenance and metadata (
Authors, repository URL publishing, embed sources), and documents verification steps in README/CHANGELOG.
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
src/IgniteUI.Blazor.Lite.csproj |
Adds NuGet authors + repository/source metadata; clarifies nullable opt-out tracking. |
README.md |
Documents release verification and links to new readiness/perf/nullable/accessibility docs. |
eng/IG.publickey.hex |
Pins the strong-name public key used for assembly identity validation. |
eng/IG.authenticode-certificates.sha256 |
Adds an allowlist of approved Authenticode signing cert fingerprints. |
eng/Check-BundleBudget.ps1 |
Implements bundle measurement + budget enforcement + release evidence reporting. |
eng/bundle-budgets.json |
Defines bundle groups/totals and budget thresholds used by the checker. |
docs/performance.md |
Publishes performance budget policy and local reproduction steps. |
docs/nullable-migration-plan.md |
Documents staged plan to re-enable nullable analysis for the shipped library. |
docs/accessibility-conformance.md |
Publishes WCAG conformance claim, scope, verification approach, and known failures. |
CHANGELOG.md |
Records new release evidence, signing/provenance changes, and breaking strong-name signing. |
.gitignore |
Ignores artifacts/ produced by release evidence jobs/scripts. |
.github/workflows/igniteui-blazor-lite-release.yml |
New multi-job release workflow with signing, provenance checks, SBOM + attestations, and release attachments. |
.github/scripts/verify-strong-name.ps1 |
Validates strong-name signing against a pinned public key (not just sn -vf). |
.github/scripts/Assert-NuspecRepository.ps1 |
Fails release if nuspec provenance metadata is missing/incorrect. |
.config/sbom-tool/dotnet-tools.json |
Pins sbom-tool via a dedicated tool manifest for the SBOM job. |
Suppressed comments (3)
.github/workflows/igniteui-blazor-lite-release.yml:244
actions/download-artifactis extracting thesigned-assembliesartifact intosrc/, but the artifact paths already start withsrc/...(src/bin/**,src/obj/**,src/wwwroot/**). This will typically createsrc/src/..., causingdotnet pack --no-buildto use the unsigned/unbuilt checkout outputs instead of the downloaded signed ones.
- name: Download signed assemblies
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: signed-assemblies
path: src
digest-mismatch: error
.github/workflows/igniteui-blazor-lite-release.yml:358
- The
evidencejob downloadsbuild-outputintosrc/, but the artifact itself containssrc/wwwroot/**. This will typically extract tosrc/src/wwwroot, whileeng/Check-BundleBudget.ps1expects assets undersrc/wwwroot(fromeng/bundle-budgets.json). That mismatch will make the budget check fail even when the build produced assets.
- name: Download build output
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: build-output
path: src
digest-mismatch: error
eng/Check-BundleBudget.ps1:151
- Same rounding issue for totals: comparing
result.RawKiB/result.GzipKiB(rounded) can let a total exceed its budget without failing the build. Use$raw/$gzipbyte totals for the enforcement condition.
if ($result.RawKiB -gt $total.maxRawKiB) {
$problems += "Total '$($total.id)' is $($result.RawKiB) KiB raw, over its $($total.maxRawKiB) KiB budget."
}
if ($null -ne $total.maxGzipKiB -and $result.GzipKiB -gt $total.maxGzipKiB) {
$problems += "Total '$($total.id)' is $($result.GzipKiB) KiB gzipped, over its $($total.maxGzipKiB) KiB budget."
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…d refactor comments
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 15 changed files in this pull request and generated 3 comments.
Suppressed comments (3)
Previously missed (2) — in code that hasn't changed since the last review.
docs/accessibility-conformance.md:34
- This states that the axe and keyboard suites currently run and gate releases, but the status section below says neither suite exists yet. Describe these layers in future tense so consumers do not mistake planned verification for completed evidence.
1. **Automated scanning.** An axe-core scan runs over every component in the Playwright integration suite, asserting the `wcag2a`, `wcag2aa`, `wcag21a`, `wcag21aa`, and `wcag22aa` rule sets. It gates pull requests and the release, and the resulting report is attached to the GitHub release as evidence.
2. **Keyboard operation.** Covered by the same suite: tab order, roving tab stops, arrow-key navigation, activation, and focus restoration.
3. **Screen reader smoke testing.** Manual, once per major release, against the matrix below.
docs/performance.md:32
- The generated files do not match exactly one pattern: for example, an
app.<hash>.bundle.jsmatches bothapp.*.bundle.jsand the later*.bundle.jscatch-all. The checker intentionally assigns the first match, so document that ordering rule instead of claiming uniqueness.
Bundle filenames are content-hashed, so budgets are expressed as patterns rather than filenames. Every produced file must match exactly one group — an asset that matches none fails the check, so a new bundle cannot enter the package without someone budgeting for it.
.github/workflows/igniteui-blazor-lite-release.yml:226
- The pack job is also placed in the NuGet publishing environment while holding
id-token: write. NuGet's OIDC policy matches repository/workflow/ref/environment claims rather than the job name, so this job can mint the same short-lived publish credential as the nominal publish-only job. Move package signing to a separate environment and keepnuget-org-publishexclusive to the final job.
environment: nuget-org-publish
permissions:
contents: read
id-token: write
#365 Pull requests are gated by dependency-review (fails on High and above). The release scans what it ships and records the report as a release asset, but stays advisory so a finding never holds up a publish. PR #365 enables nullable analysis outright and makes the staged migration plan moot, so the doc and its references are removed and the csproj nullable block is left exactly as master has it to keep that PR merging cleanly.
…ng the release evidence
There was a problem hiding this comment.
🟡 Changes recommended
Package metadata is inconsistent and two stated release-integrity assertions are not fully enforced.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 25/27 changed files
- Comments generated: 4
- Review effort level: Balanced
| if-no-files-found: error | ||
|
|
||
| pack: | ||
| name: Pack and sign package |
There was a problem hiding this comment.
Pack and sign should probably be separate jobs too, since one creates an artifact, while the other signs it.
Same concern and reasoning as here: IgniteUI/Infragistics.QueryBuilder.Executor#28 (comment)
There was a problem hiding this comment.
I've discussed this with Damyan and we deem it safe enough to bundle these operations in the same job.
| retention-days: 1 | ||
| if-no-files-found: error | ||
|
|
||
| # Holds the strong-name key, but no OIDC token, no Key Vault access and no publishing rights. |
There was a problem hiding this comment.
If this split between the jobs Build web assets and Build library is because of these:
- **CI-07** (`repository-wide`) Untrusted build execution is separated from privileged signing/publishing.
- **CI-08** (`repository-wide`) Signing/publishing consumes a verified immutable artifact.
It might not be necessary since the strong-name key doesn't seem to be considered privileged signing/publishing.
- doesn't do publishing
- doesn't establish client-trusted release identity or any type of security, also doesn't seem to necessarily be a secret.
I think we can skip this split. Just do a single build for both.
There was a problem hiding this comment.
You are correct - I'll put the build-assets job back into the build job.
The split was done based on an earlier Copilot review comment that I tentatively agreed with at the time.
| retention-days: 30 | ||
| if-no-files-found: error | ||
|
|
||
| # Advisory by design. A finding is annotated and attached to the release as evidence, but never |
There was a problem hiding this comment.
Not sure if this will fully satisfy this one:
- **SEC-07** (`repository-wide`) No known unpatched High/Critical vulnerabilities at release time.
Since in theory a dependency that was clean when merged may have a new High/Critical advisory disclosed against it later — with no PR touching it.
Sounds like this one needs to fail if it finds a vulnerability.
While the one on PR that currently fails, may not be necessary. While reading on it here: https://learn.microsoft.com/en-us/nuget/concepts/auditing-packages#running-nuget-audit-in-ci
By using conditions to selectively cause NuGet Audit warnings to fail a restore, you can have a dedicated pipeline to check packages for known vulnerabilities, while preventing new security advisories from blocking your bug fixes at inconvenient times. Keeping NuGet Audit warnings enabled for local builds allows developers to get a non-blocking notification about new security advisories and can encourage upgrading package versions to fix the vulnerabilities more quickly than waiting for someone to check the audit pipeline status.
It sounds like failure on PR is optional, but failure on release is not.
There was a problem hiding this comment.
@damyanpetev , I think we discussed this earlier. Given the SEC-07 that Maya quotes, I think we ultimately decided that we just want a warning during a release run.
Can you give us a reminder here please?
kdinev
left a comment
There was a problem hiding this comment.
@turbobobbytraykov Are you going to include a different release pipeline for IgniteUI.Blazor.Templates, or is this coming later (not part of this PR)?
@damyanpetev , say the word. |
|
Ran the readiness assessment skill and here are the open defects relevant to this PR: Complete targeted scorecard —
|
| Requirement ID | Requirement | Evidence | Maintainer action | Reviewer follow-up |
|---|---|---|---|---|
| CI-05 | Dependency scans and release checks are required gates. | [EV1-c23b5da00c836eceefb1deea21315e8d53f187efa96a89b7cc1025e84baa4f24] [EV1-eb20aea78c6cb74f428b015881dcc3c050a9577e2df5b914544d0b46e53cc5d6] | Make the dependency scan a required gate: add Dependency Review to the required checks on master, and make release-time findings block the publish. | - |
| CI-06 | Default and release refs require appropriate review/checks. | [EV1-589ce7fd1a013ab6f70f73c47514a0b93ecf5d8e368bfc1b9ef9d74ee755425a] [EV1-c23b5da00c836eceefb1deea21315e8d53f187efa96a89b7cc1025e84baa4f24] | Require review or checks on the refs that can publish: add a deployment branch or tag policy to the nuget-org-publish environment, or protect the release refs directly. | - |
Evidence ledger — records cited above
| Evidence ID | Claim | Reproduction/source |
|---|---|---|
| EV1-589ce7fd1a013ab6f70f73c47514a0b93ecf5d8e368bfc1b9ef9d74ee755425a | The nuget-org-publish environment that carries Key Vault signing and NuGet.org publishing authority has no protection rule and no deployment branch policy, the only ruleset targets the default branch for Copilot review, and release 0.1.2-alpha.10 was published from the unprotected branch btraykov/release-workflow-refactoring. | gh api repos/IgniteUI/igniteui-blazor/environments |
| EV1-c23b5da00c836eceefb1deea21315e8d53f187efa96a89b7cc1025e84baa4f24 | Required status checks on master are exactly CodeQL and Build and Validate, alongside one approving review with stale-review dismissal, so the Dependency Review job that runs on pull requests with fail-on-severity high is not a required check and cannot block a merge. | gh api repos/IgniteUI/igniteui-blazor/branches/master/protection |
| EV1-eb20aea78c6cb74f428b015881dcc3c050a9577e2df5b914544d0b46e53cc5d6 | The release workflow runs the npm and webpack asset build in a job with no secret, environment or OIDC token, holds the strong-name key in a separate job with no publish rights, performs Key Vault signing and NuGet.org publishing through azure/login and NuGet/login OIDC in later stages, and re-verifies the package digest at pack, pre-attestation, pre-publish and release-attachment boundaries with digest-mismatch set to error on artifact downloads. | .github/workflows/igniteui-blazor-lite-release.yml |
…b.com/IgniteUI/igniteui-blazor into btraykov/release-workflow-refactoring
Goal
The release path was one unauthenticated job — checkout, build, pack, push to NuGet.org — with no
signing, no SBOM, no supply-chain evidence. This branch rebuilds it so a published GitHub release
produces a strong-named, Authenticode-signed, NuGet-signed
IgniteUI.Blazor.Litepackage withSPDX 2.2/3.0 + a merged CycloneDX SBOM, three attestations, and an advisory NuGet+npm dependency
scan — every job scoped to only the permissions and secrets it uses.
Decisions
build→sign-assemblies→pack→ (dependency-scan,sbom) →publish→attach-to-release.buildrunsnpm ci+ webpack and holds the strong-name key in the same job (Release buildenv). Per Microsoft's guidance, a strong-name key is not a security boundary, so isolating it from the npm toolchain in a separatebuild-assetsjob (an earlier iteration of this branch) bought nothing worth the extra job — reverted per review. Onlysign-assemblies/pack/publishget Key Vault + OIDC (nuget-org-publishenv).publishis the only job that can push and sparse-checks out just.github/scripts+ the cert pin.packrecorded (Get-PackageDigest.ps1)..github/scripts/; byte-identical toIgniteUI.Blazor.GridLitewhere shared.eng/IG.publickey.hex). The packed.nupkgis re-validated on both strong-name and Authenticode (Assert-PackageSignatures.ps1) —dotnet pack --no-buildonly re-zipsbin/. NuGet signature is certificate-fingerprint-pinned (Assert-NuGetSignature.ps1).packruns an explicitdotnet restorebeforedotnet pack --no-build --no-restore— without it pack failsNETSDK1127(targeting packs absent on the pack runner). Addresses Release pack job can fail with NETSDK1064: it packs on a runner that never restored #388.Publish-NuGetPackage.ps1refuses to overwrite an existing NuGet.org version (no--skip-duplicate).packpassesRepositoryUrl/RepositoryCommitexplicitly.New-Sbom.ps1emits SPDX 2.2 + 3.0 from onesbom-toolrun (two runs disagreed on ClearlyDefined data and cross-detected each other's manifest); retries while licence coverage improves, fails only on the 0-of-N ClearlyDefined-outage signature.dotnet-CycloneDX(--exclude-dev) only sees the.csproj; the.nupkgalso ships the webpack bundle +igniteui-webcomponentstheme CSS, covered byNew-NpmCycloneDxSbom.ps1(--omit dev, pinned devDependency).Merge-CycloneDxSbom.ps1combines them in pure PowerShell (cyclonedx-cliis GitHub-binary-only;cyclonedx-librarycan't deserialize JSON). Both halves pinned to spec 1.6.-ComponentScanExclusion '**/artifacts/**'keeps the packed.nupkg(under-b, inside the repo-root-bc) from being detected as its own dependency;Assert-Sbom.ps1 -PackageIdthen fails the release if any SPDX 2.2/3.0 package entry outside the document root carries the released id, so a dropped or mis-globbed exclusion cannot pass unnoticed.sbom-toolhas no dev/runtime filter, so SDK targeting packs,ILLink.Tasks(from<IsTrimmable>) and analyzers stay in the SPDX .NET section by design;--exclude-devkeeps the CycloneDX .NET half clear of them.purl+licenses(-RootLicenseExpression MIT, never inherited from the npm side). Inputmetadata.componentsub-roots are dropped fromcomponents[]; the root edge points straight at both ecosystems' first-level deps.metadata.tools.components[]is rebuilt from both inputs + aMerge-CycloneDxSbom.ps1entry (-MergeToolVersion= release SHA). A prune pass drops edges whosebom-refno component declares (cyclonedx-npm nested-path artifacts).Assert-CycloneDxSbom.ps1fails ifpkg:nuget/*orpkg:npm/*is absent..nupkg.sha256,.cdx.json.sha256) are LF-terminated<hash> <name>— CRLF / bare-hash forms brokesha256sum -coff Windows.dependency-scanis advisory (dotnet list package --vulnerable+npm audit --omit=dev); the blocking PR gate isci.yml'sdependency-review..config/dotnet-tools.jsonis the single tool manifest.evidencejob andeng/Check-BundleBudget.ps1/eng/bundle-budgets.json. WASM bundle-size budgets may return later as an npm build-time check instead of a release-workflow gate.Validation
Last fully-completed run (pre-dates this session's job merge and evidence removal):
0.1.2-alpha.10 /
run 34577931933. All 9
jobs (the pre-merge graph) green.
Assert-Sbom.ps1ran with-PackageId, passed with 0 self-entries; the release SPDX has noIgniteUI.Blazor.*package entry (68 packages).gh release download: single synthetic root,purl+licenses: MIT,metadata.tools.components[]of 5, noISC, 0 danglingdependencies[]refs, 58 components (44 NuGet, 14 npm) — the npm half now includestslib/@floating-ui/core(apackage-lock.jsonregen closed the earliercyclonedx-npmgap; the dependency-edge prune remains as defense-in-depth for future recurrences).sha256sum -cOK on both sidecars;performance-report.mdrendered correctly; SPDX 2.2/3.0 both recorded 68 packages; all three attestations bound to the published.nupkgdigest.Assert-Sbom.ps1's self-reference gate verified against a clean SPDX (passes), an injected self-entry in both 2.2 and 3.0 (fails, both reported), and a wrong--PackageIdnegative control (passes)..github/scripts/*.ps1pass AST parsing; the workflow parses as valid YAML and its job graph resolves cleanly (build→sign-assemblies→pack→ (dependency-scan,sbom) →publish→attach-to-release).Open
build/build-assetsmerge and theevidence/bundle-budget removal — have not yet been through a release run. Local checks only: YAML parses, job graph has no danglingneeds, no leftoverbuild-assets/web-assets/evidence/budget references anywhere in the repo. Needs a release to confirm the combinedbuildjob produces identicalbuild-outputand thatattach-to-releasestill succeeds without the evidence artifact.Merge-CycloneDxSbom.ps1's JSON merge is hand-written. Covered: real BOMs + local synthetic edge cases. Not covered:vulnerabilitiesdata, duplicatebom-refacross the two inputs.ILLink.Tasks/ analyzers remain in the SPDX .NET section — accepted (sbom-toolhas no CLI filter; a post-filter was deliberately not added).cyclonedx-npmgains 1.7 support.