Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- Clarified immutability guidance for emitted CTRF report artifacts ([#55](https://github.com/ctrf-io/ctrf/pull/55)).
- Clarified `tags` as simple keyless classifications and `labels` as structured key-value test metadata ([#54](https://github.com/ctrf-io/ctrf/pull/54)).
- Allowed non-empty arrays of strings, numbers, and booleans as label values ([#54](https://github.com/ctrf-io/ctrf/pull/54)).
26 changes: 26 additions & 0 deletions spec/ctrf.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,26 @@ As long as the document structure and semantics are respected, CTRF does not con

---

### 2.11. Immutable Report Artifacts

CTRF treats an emitted report as an immutable artifact.

A CTRF document represents a snapshot of test results at the time it is produced. Implementations are free to build or transform documents internally before emission, but once a CTRF document has been written, published, or otherwise made available as an artifact, it is treated as a fixed record.

Immutability supports:

- auditability of test results
- reproducibility of analysis
- stable caching and artifact storage
- compatibility with hashing, signing, and deduplication
- clear separation between raw results and derived results
Comment thread
Ma11hewThomas marked this conversation as resolved.

When a tool or system performs additional processing after initial report generation — such as merging shard reports, adding metadata, or computing insights — the design intent is that this produces a new CTRF document and, when `reportId` is used, assigns a different `reportId` value, rather than modifying the original.

This distinction between the act of constructing a report and the resulting artifact as an immutable record is central to how CTRF is designed to be used in pipelines, artifact stores, and analysis systems.

---

## 3. Terminology

The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**,
Expand Down Expand Up @@ -405,6 +425,8 @@ A unique identifier for this report instance.
`reportId` is OPTIONAL.
If present, it MUST be a valid UUID as defined in [RFC4122].

Each emitted CTRF document represents a distinct report instance. If a new CTRF document is produced through post-processing, it SHOULD use a different `reportId` value than the source document when `reportId` is used.

---

### 5.4. `timestamp`
Expand Down Expand Up @@ -1770,6 +1792,8 @@ Producers SHOULD:
- provide timestamps
- provide environment/build metadata
- provide diagnostics when available
- treat emitted CTRF documents as immutable artifacts
- emit a new CTRF document when performing post-processing after initial report generation and, when `reportId` is used, assign a different `reportId` value rather than modifying an emitted CTRF document

---

Expand All @@ -1784,6 +1808,8 @@ Consumers:
- MUST NOT require optional fields to be present
- MUST NOT assume the presence of `insights`
- SHOULD compute insights when sufficient historical data is available
- SHOULD assume that CTRF documents are immutable once emitted
- MUST NOT rely on in-place modification of existing reports

---

Expand Down
Loading