Add immutability principle

[Ma11hewThomas]

Summary

Adds an immutability principle to the CTRF specification.

The core idea is that an emitted CTRF report should be treated as an immutable artifact: a fixed snapshot of test results at the time it was produced. Producers can still build or transform reports internally before emission, but once a report has been written, published, uploaded, or otherwise made available as an artifact, consumers and downstream systems should treat it as stable.

Why This Matters

Immutability gives CTRF reports stronger artifact semantics for CI systems, report stores, dashboards, and downstream analysis.

It supports:

• auditability of test results
• reproducibility of analysis
• stable caching and artifact storage
• compatibility with hashing, signing, and deduplication
• a clearer separation between raw emitted results and derived reports

This is especially important for workflows that merge shards, enrich report metadata, compute insights, or otherwise post-process test output after initial report generation.

Behavior Clarified

This PR clarifies that post-processing should produce a new CTRF document rather than mutate an already-emitted one.

Examples of post-processing include:

• merging shard reports
• adding metadata after initial generation
• computing derived insights
• producing a derived or enriched report for another system

When a new CTRF document is produced through that kind of post-processing, it should receive a new reportId.

Implementation Details

This PR updates spec/ctrf.md only, plus the changelog.

Spec changes:

• Adds Section 2.11 Immutable Report Artifacts
• Adds a reportId note in Section 5.3
• Adds producer conformance guidance:
  • producers should treat emitted CTRF documents as immutable artifacts
  • producers should emit a new CTRF document with a new reportId when post-processing after initial report generation
• Adds consumer conformance guidance:
  • consumers should assume CTRF documents are immutable once emitted
  • consumers must not rely on in-place modification of existing reports

Changelog changes:

• Adds an Unreleased entry for the immutability guidance

Compatibility

This is a documentation/specification clarification. It does not change the JSON Schema and does not alter document validation.

The change strengthens the intended artifact model without invalidating existing CTRF documents.

Validation

• jsonschema validate schema/ctrf.schema.json examples/*.json
• jsonschema test tests/ctrf.test.json
• jsonschema test tests/normative
• jsonschema test tests/informative

[Copilot]

Pull request overview

This PR updates the CTRF specification to clarify an “immutability” principle: once a CTRF report is emitted as an artifact, it should be treated as a stable snapshot, and post-processing should produce a new document (with a new reportId) rather than modifying an existing artifact.

Changes:

• Adds Design Principle §2.11 describing emitted CTRF reports as immutable artifacts and positioning post-processing as producing a new document.
• Clarifies reportId guidance to emphasize distinct report instances for post-processed outputs.
• Extends producer/consumer conformance guidance to discourage mutation of emitted reports and reliance on in-place modification.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
spec/ctrf.md	Adds an immutability principle, reinforces reportId semantics for derived reports, and updates producer/consumer conformance guidance.
CHANGELOG.md	Records the immutability guidance under Unreleased changes.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.