docs: document summary definitions on custom attestation types

[dangrondahl]

Closes #364

What

Documents summary on custom attestation types — the ordered list of named jq expressions that Kosli renders as labeled rows on the attestation detail page instead of only raw JSON.

The issue was filed when summary was API-only and recommended holding back the authoring instructions. Both surfaces have since shipped (--summary / --summary-json on kosli create attestation-type, and the summary attribute on kosli_custom_attestation_type), so all four open tasks are covered.

Changes

• getting_started/attestations.md — new #### Summaries subsection in the Custom accordion: what a summary is, the --summary CLI flags, the JSON form, and the behavior worth knowing (part of the versioned type definition, evaluated at display time, invalid jq rejected at creation, N/A fallback, URLs render as links, one summary group per element for array payloads).
• tutorials/attest_custom.md — step 1 now creates the coverage-report type with --summary "Coverage=.coverage" / --summary "Tool=.tool", so the walkthrough ends on a readable summary rather than raw JSON.
• administration/managing_custom_attestation_types/overview.md — summary added to the "Each type can include" list (and the "at least one of the two" sentence reworded accordingly), plus a With a summary Terraform example using jsonencode() and the file() variant.

No new pages, so no config/navigation.json change.

Verification

• mint broken-links — the only reported link is the pre-existing /getting_started/service-accounts in tutorials/working_with_controls.mdx, untouched here.
• Rendered all three pages against mint dev: they compile, and the new #summaries heading inside the accordion does get an anchor (id="summaries"), which the tutorial and Terraform page link to.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
kosli	🟢 Ready	View Preview	Sep 3, 2026, 11:16 AM

[claude]

Claude finished @dangrondahl's task in 2m 56s —— View job

---

Doc quality review

Reviewed all three files at b823c4f. Anchors, links and structure check out: #summarizing-custom-attestations resolves as an ## heading with a TOC entry, both cross-links point at it, #annotating-attestations resolves, all internal targets exist, no relative links, no new pages so no config/navigation.json or config/redirects.json work needed.

Two findings.

terraform-reference/resources/custom_attestation_type.mdx — the link-rule sweep stopped one file short

Location: terraform-reference/resources/custom_attestation_type.mdx:51 and :189

This PR narrows the rule to "a string beginning with http:// or https://" on both pages it touches, and the PR description names the Terraform page as a known leftover. But that page is the one administration/managing_custom_attestation_types/overview.md:138 sends Terraform users to under Reference — so the reader most likely to be writing a summary block is also the one most likely to read the superseded phrasing:

• :51 — code comment: # A value that is a valid URL renders as a clickable link.
• :189 — attribute description: values that are valid URLs render as links

terraform-reference/ is agent-synced from kosli-dev/terraform-provider-kosli, not deterministically regenerated, so an edit here does stick until the next sync — unlike client_reference/kosli*.md. Fixing the page and the provider source together is the durable version; fixing the page alone still helps today. Two related gaps in the same file while you're there: line 51's comment says labelled (British — that page is excluded from Vale, so it isn't caught), and the "They can include:" list at lines 9–14 enumerates only schema and JQ rules, so it never mentions summary even though the resource supports it. overview.md:12–16 now lists all three.

Fix this →

getting_started/attestations.md:419 — unverified comparative claim about annotation values

Posted inline. Short version: the narrow summary rule is well sourced, but "stricter than the rule for annotation values" is a claim about annotations whose only evidence is line 246 of the same page — prose written before the hardening commit. Dropping the comparison costs nothing; keeping it commits the docs to a difference nobody checked.

Upstream — does not block this merge

client_reference/kosli_create_attestation-type.md:81–85 ships an example titled "create/update a custom attestation type with no schema no evaluation rules" — bare kosli create attestation-type customTypeName. That contradicts administration/managing_custom_attestation_types/overview.md:18 (reworded in this PR) and terraform-reference/resources/custom_attestation_type.mdx:14, which both state the API requires at least one of schema or jq rules. One of the two is wrong, and the question now matters more than it did: with summary in the picture, a reader reasonably asks whether a summary-only type is legal, and these pages answer differently.

Pre-existing and only reachable upstream — kosli-dev/cli → cmd/kosli/createAttestationType.go. Also there: the --summary example accordions at lines 97–112 have no \ line continuations, so they aren't copy-pasteable (a long-standing trait of every example block on those generated pages, not new to --summary).

---

1 Improvement in this repo, 1 Improvement inline, 1 upstream note. Nothing blocking — the versioning and link-rule corrections in 09e392f/b823c4f landed the two substantive items from the previous round.
· docs/364-attestation-type-summaries

[dangrondahl]

All five inline threads answered and pushed (b5e...see latest commit). Replying here to the one finding that had no inline anchor, plus what came out of verifying the rest.

The two substantive items were both real.

The versioning contradiction resolved in favor of the attestations.md sentence: attestation.py resolves find_by_id(self.type_id).get_versioned_type(self.type_version) before evaluating, so the detail page renders against the version the attestation was reported against. Applied the suggested <Note> rewording and made the pinning explicit on both pages.

The anchor concern was not a maybe — a headless-Chrome load of #summaries as a fresh navigation left every accordion at aria-expanded="false", byte-identical to a no-hash load. The content is now a top-level ## Summarizing custom attestations section with a pointer left in the Custom accordion, and both cross-links updated.

Checking those also caught something the review didn't: "a value that is a valid URL renders as a link" was too broad. The hardening commit on kosli-dev/server#6428 requires a string value with an http/https scheme, deliberately, so that non-string values don't 500 and javascript:/data: can't become clickable — it supersedes the is_valid_url annotation convention the issue cited. Both pages now say so. Note the generated terraform-reference page still carries the loose phrasing; that's a provider-docs fix, not one for this PR.

On the "reference material in a getting-started page" finding — I mostly disagree, and changed nothing for it.

Of the three overlaps named: URL-as-link is not in client_reference/kosli_create_attestation-type.md (only in the Terraform reference, where it's imprecise); split-on-first-= and the checklist fallback are each one clause of prose, not bullets, and both are load-bearing where they sit — the first is a real trap for anyone writing == in a jq expression, the second is the sentence that motivates the whole section. The six bullets themselves are behavioral (versioning, display-time evaluation, creation-time rejection, N/A, link rules, array payloads), which the CLI reference does not and should not carry.

The drift risk is worth naming, though: the generated CLI page is regenerated from the CLI repo, so if the flag semantics change the hand-written section won't follow. That's an argument for keeping the flag mechanics thin here rather than for trimming the behavior, which is what I've done.

Also noted and agreed on the British "labelled" in the generated pages — upstream fix, out of scope here.

[dangrondahl]

Both inline threads answered and pushed in b823c4f. Replying here to finding 3, which had no inline anchor, and to the "Not verified" list — I had the server checkout to hand this round, so the load-bearing bullets are no longer taken on trust.

Finding 3 (split code span) — fixed. The span now sits on one line:

order, with the values taken from each attestation's own data. Re-running
`kosli create attestation-type` for a name that already exists updates that type rather than
creating a second one — see the note on versioning below.

On my own earlier claim, which was wrong. I said the #6428 hardening "supersedes the is_valid_url annotation convention". It doesn't — annotations still render through is_valid_url, and only the summary path was tightened. Details in the inline thread; the net effect is that the fix was to drop the "as annotation values do" equivalence, not to change either rule's description.

The five product-behavior claims the review couldn't check are all confirmed:

Claim	Source
Renders against the pinned type version	attestation.py:486 — find_by_id(self.type_id).get_versioned_type(self.type_version)
Invalid jq rejected at creation, not render	custom_attestation_summaries.py:19 — a field_validator on expression calling jq.compile, raising 400
null or a failed expression renders N/A	evaluate returns None on both paths; custom_attestation.html:24 renders N/A for row.value is none, per-row, so the rest of the summary survives
One summary group per top-level array element	custom_attestation_types.py:114 — elements = data if isinstance(data, list) else [data]
Reads like the built-in Sonar/Snyk/JUnit attestations	sonar_attestation.html uses the same labeled <td><span class="opacity-50 text-nowrap"> row shape

Checking the fourth one turned up a small inaccuracy of my own: numbering is conditional on summary_groups | length > 1, so a top-level array with exactly one element renders a bare Summary:, not Summary 1:. The bullet now says numbering appears "when there is more than one".

There is also a whole-summary failure state I've deliberately left undocumented — the SUMMARY_UNAVAILABLE sentinel renders a "Summary unavailable" banner with the checklist beneath it. That only fires if the type or version lookup itself throws, which is an internal error rather than something an author can cause or act on.

mint broken-links is clean apart from the pre-existing /getting_started/service-accounts in working_with_controls.mdx.

Two follow-ups belong in the provider repo, not here: terraform-reference/resources/custom_attestation_type.mdx:189 describes summary linkification as "values that are valid URLs", which is now known-wrong, and spells it "labelled". That page is generated, so I haven't touched it — say the word and I'll file it against the provider.

[vidhu-balad]

Non-blocking comment: The two flags --summary and --summary-json are mutually exclusive, that is the phrasing pattern used across the docs. This change mentions it with "instead". Makes sense but could be updated to stay consistent.