Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions docs/testing/dcp-test-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ The TCK framework is designed to run as part of a **CI pipeline** (Docker image
- **DCP §** — the spec section(s) containing the normative statement(s) the test traces to. Only sections with actual MUST/SHOULD/MAY text binding the tested behavior are listed. `—` means no normative statement exists; the section numbering follows the rendered v1.0.1 spec (§4 Base Concepts, §5 Verifiable Presentation Protocol, §6 Credential Issuance Protocol, Appendix A DCP Profiles), confirmed against the section numbers the TCK itself uses in its `@DisplayName`s.
- **TCK Test** — the TCK test method name (from the `dcp-testcases` source). Blank means no TCK test exercises the case: it is one of the enhanced tests to be added per §1.3.
- **Tag rule** — the TCK tests only normative requirements (all its tests are `@MandatoryTest`), so cases tagged SEC — and untested SHOULD/MAY behaviors — have no TCK test by construction. Notes cite "Tag rule" for these.
- **IH** — status of eclipse-edc IdentityHub/IssuerService for the full case as specified, verified against the IdentityHub/IssuerService codebase and test suite on `main` (as of 2026-08-21). Existing TCK tests are assumed to pass. Cases whose Notes cite `eclipse-edc/IdentityHub#1063` are fixed on that pull request and are green only once it merges.
- **IH** — status of eclipse-edc IdentityHub/IssuerService for the full case as specified, verified against the IdentityHub/IssuerService codebase and test suite on `main`, up to and including [#1063](https://github.com/eclipse-edc/IdentityHub/pull/1063) (merged 2026-08-29). Existing TCK tests are assumed to pass.
- **DIV** — status of the SAP DIV Credential Service; to be filled in during verification.
- **Notes** — partial-coverage caveats and explanation where a status needs it.

Expand Down Expand Up @@ -146,9 +146,9 @@ Note on the Bearer-header requirement (TOK-02/03): within DCP §6 the MUST is st
| TOK-06 | `exp` in the past (beyond leeway) | 401 | MUST | 4.3, 4.3.3 | `*_tokenExpired` | ✅ | | TCK also tests `iat` in the future (`*_iatInFuture`), an extra variant not in this suite |
| TOK-07 | `nbf` in the future (beyond leeway) | 401 | MUST | 4.3.3 | `*_nbfViolated` | ✅ | | |
| TOK-08 | Signature by a key not present in the `sub` DID document | 401 | MUST | 4.3.3 | `*_tokenSignedWithWrongKey` | ✅ | | |
| TOK-09 | `kid` referencing a nonexistent verification method | 401 | MUST | 4.3.3 | | ✅ | | No TCK test — verified against source. The upstream key resolver fails to resolve a `kid` naming no verification method, and eclipse-edc/IdentityHub#1063 makes that answer binding by requiring the `kid` to belong to the `iss` DID |
| TOK-09 | `kid` referencing a nonexistent verification method | 401 | MUST | 4.3.3 | | ✅ | | No TCK test — verified against source. The upstream key resolver fails to resolve a `kid` naming no verification method, and the holder's endpoints require the `kid` to belong to the `iss` DID, which makes that answer binding (#1063) |
| TOK-10 | Signing key lacks the `capabilityInvocation` relationship (e.g. listed only under `authentication`) | 401 — signature validity alone is insufficient | MUST | 4.3.3 | | ⚠️ | | IH cannot express this: the `capabilityInvocation` relationship has no property on the upstream DidDocument model, so it can neither be published nor enforced. Blocked on the Connector |
| TOK-11 | No `kid` header and the DID document contains more than one verification method | 401 | MUST | 4.3.3 | | ✅ | | No TCK test — verified against source. The upstream key resolver rejects an absent `kid` against a multi-key DID document, and eclipse-edc/IdentityHub#1063 rejects a token without a `kid` outright on the holder's endpoints |
| TOK-11 | No `kid` header and the DID document contains more than one verification method | 401 | MUST | 4.3.3 | | ✅ | | No TCK test — verified against source. The upstream key resolver rejects an absent `kid` against a multi-key DID document, and the holder's endpoints reject a token without a `kid` outright (#1063) |
| TOK-12 | Replay of a previously used `jti` | 401 | MUST | 4.3, 4.3.3 | `*_jtiAlreadyUsed` / `*_jtiUsedTwice` | ✅ | | IH requires `edc.iam.accesstoken.jti.validation=true` |
| TOK-13 | `sub` DID not resolvable | 401 | MUST | 4.3.3 | | ⚠️ | | No TCK variant exists — verified against source. IH's own tests cover an unresolvable `sub` DID on the Storage API and the Credential Offer API; the remaining endpoints are unverified |

Expand All @@ -164,7 +164,7 @@ Note on the Bearer-header requirement (TOK-02/03): within DCP §6 the MUST is st
| CS-STOR-02 | Same, for every DCP profile the CS claims to support | 2xx, credential usable | MUST | A.2 | `cs_06_05_01_credentialMessage` (vc11 only) | ✅ | | `vc20-bssl/jwt` never generated by the TCK; IH's own tests cover `vc20-bssl/jwt` storage |
| CS-STOR-03 | Message missing required fields (`@context`, `type`, `issuerPid`, `holderPid`, `status`) | 400 | MUST | 6.5.1 | `cs_06_05_01_credentialMessage_invalidBody` | ✅ | | TCK variant removes `holderPid` only; IH's own tests cover the remaining field omissions |
| CS-STOR-04 | `status` outside `ISSUED`/`REJECTED` | 400 | MUST | 6.5.1 | `cs_06_05_01_credentialMessage_invalidStatus` | ✅ | | |
| CS-STOR-05 | `status=REJECTED` (with `rejectionReason`) | 2xx; nothing stored; the request is subsequently reported as rejected/failed by the CS | MUST | 6.5.1 | | ✅ | | eclipse-edc/IdentityHub#1063: the holder fails the request on a rejection and records the issuer's `rejectionReason`, which is carried on the message as an OPTIONAL property |
| CS-STOR-05 | `status=REJECTED` (with `rejectionReason`) | 2xx; nothing stored; the request is subsequently reported as rejected/failed by the CS | MUST | 6.5.1 | | ✅ | | The holder fails the request on a rejection and records the issuer's `rejectionReason`, which travels on the message as an OPTIONAL property (#1063) |
| CS-STOR-06 | `status=ISSUED` with empty `credentials` array | Accepted as a no-op (2xx, nothing stored, pending request unaffected) | SEC | — | | ✅ | | Tag rule; `credentials` is OPTIONAL in §6.5.1, empty-array semantics unstated. IH accepts the message as a no-op and leaves the pending request untouched; covered by its own tests |
| CS-STOR-07 | `holderPid` that matches no pending request | 4xx, nothing stored | MUST | 6.5, 6.5.1 | | ✅ | | No TCK test — verified against source. IH's own tests cover this |
| CS-STOR-08 | Credential type or format that was not requested under this `holderPid` | 4xx, nothing stored | SEC | — | | ✅ | | Tag rule; IH's own tests cover both variants |
Expand Down Expand Up @@ -243,7 +243,7 @@ The TCK's `issuance.cs` package tests only the CS's inbound APIs (Storage, Offer
| ID | Stimulus | Expected | Tag | DCP § | TCK Test | IH | DIV | Notes |
|---|---|---|---|---|---|---|---|---|
| IS-META-01 | Fetch metadata | 200 `IssuerMetadata`: `issuer` = issuer DID, `credentialsSupported` array of `CredentialObject`s | MUST | 6.7, 6.7.1 | `is_6_7_issuerMetadata` | ✅ | | Asserts `type`, `issuer` DID, non-empty array only |
| IS-META-02 | Inspect each `CredentialObject` | Contains **all** optional properties (`credentialType`, `bindingMethods`, `credentialSchema`, `profile`, `issuancePolicy`, `offerReason`) with meaningful, well-formed values | MUST | 6.7.1 | | ✅ | | eclipse-edc/IdentityHub#1063: `credentialSchema` is published from the credential definition's schema URL, and the issuance policy id is derived from the definition rather than generated per request, which also makes the object stable (IS-META-03) |
| IS-META-02 | Inspect each `CredentialObject` | Contains **all** optional properties (`credentialType`, `bindingMethods`, `credentialSchema`, `profile`, `issuancePolicy`, `offerReason`) with meaningful, well-formed values | MUST | 6.7.1 | | ✅ | | `credentialSchema` is published from the credential definition's schema URL, and the issuance policy id is derived from the definition rather than generated per request, which also makes the object stable across fetches (IS-META-03) (#1063) |
| IS-META-03 | Fetch twice | `CredentialObject.id`s are stable — clients reference and cache them | MUST | — (implied by the resolution MUSTs in 6.4.1, 6.6.1) | | ✅ | | No explicit normative statement — spec tightening candidate; IH ids equal credential-definition ids, stable by construction |
| IS-META-04 | Issuer with no credential types configured | 200 with empty `credentialsSupported` | SHOULD | — | | ✅ | | Tag rule; `credentialsSupported` is OPTIONAL in §6.7.1. IH returns 200 with an empty array; covered by its own tests |

Expand All @@ -261,7 +261,7 @@ Offer emission (IS-OFF-01/02) depends on the issuer trigger defined in §2.2.
| IS-DELIV-06 | Delivery retry after an ambiguous outcome | Re-delivery carries the same `issuerPid`/`holderPid` so a compliant holder can deduplicate | SEC | — | | ✅ | | Tag rule; a re-delivery is sent from the same issuance process, so both pids are unchanged, and IH's holder side deduplicates it (CS-STOR-13); both covered by its own tests |
| IS-OFF-01 | Issuer sends an offer | Well-formed `CredentialOfferMessage` to the holder's `/offers` endpoint: `issuer` DID + non-empty `credentials`, each entry resolvable against the issuer's own metadata | MUST | 6.6.1 | | ✅ | | TCK has no trigger to make the IS emit an offer; IH's own e2e covers the full offer round trip |
| IS-OFF-02 | SI token on the offer | `aud` = holder DID | MUST | 4.3 | | ✅ | | Same; holder-side verification passes in IH's e2e |
| IS-OFF-03 | Key rotation / re-issuance scenarios, if supported | Offers carry `offerReason` = `reissue` / `proof-key-revocation` | MAY | 6.6.2, 6.9.1, 6.9.2 | | ✅ | | Tag rule; offers reused the metadata objects verbatim, so every offer claimed `reissue`. eclipse-edc/IdentityHub#1063 takes the reason from the caller of the Credential Offer API, defaulting to `reissue` |
| IS-OFF-03 | Key rotation / re-issuance scenarios, if supported | Offers carry `offerReason` = `reissue` / `proof-key-revocation` | MAY | 6.6.2, 6.9.1, 6.9.2 | | ✅ | | Tag rule; offers previously reused the metadata objects verbatim, so every offer claimed `reissue`. The reason now comes from the caller of the Credential Offer API, defaulting to `reissue` (#1063) |

---

Expand All @@ -285,9 +285,9 @@ The harness plays the verifier: it obtains a VP access token from the CS and que
| CS-PRES-08 | Scope query by credential type with a matching access token | 2xx; VP containing the credential of that type | MUST | 5.4.1.2, 5.4.2 | `cs_05_04_01_02_scopeByTypeRequest` | ✅ | | |
| CS-PRES-09 | Access token authorizes more scopes than the query requests | 2xx with only the queried subset | MUST | 5.4.1.2 | `cs_05_04_01_02_lessScopesThanAuthorizedByTypeRequest` | ✅ | | |
| CS-PRES-10 | Scope escalation: query for a scope the access token does not authorize | 4xx or reduced response; no unauthorized data returned | MUST | 5.4.1.2 | `cs_05_04_01_02_invalidScopeEscalationRequest` | ✅ | | |
| CS-PRES-11 | Query with scope alias `org.eclipse.dspace.dcp.vc.id:<credential-id>` and matching access token; negative: nonexistent credential ID | 2xx with exactly that credential; negative → 2xx with empty/reduced `presentation` array, no error leakage | MUST | 5.4.1.2.2, 5.4.1.2 | | ✅ | | The `vc.id` alias MUST be supported; untested by the TCK. Implemented in eclipse-edc/IdentityHub#1063, including ids that themselves contain separators, and scope escalation is caught for the alias as it is for types |
| CS-PRES-11 | Query with scope alias `org.eclipse.dspace.dcp.vc.id:<credential-id>` and matching access token; negative: nonexistent credential ID | 2xx with exactly that credential; negative → 2xx with empty/reduced `presentation` array, no error leakage | MUST | 5.4.1.2.2, 5.4.1.2 | | ✅ | | The `vc.id` alias MUST be supported; untested by the TCK. IH implements it, including ids that themselves contain separators, and scope escalation is caught for the alias as it is for types (#1063) |
| CS-PRES-12 | Three credentials of one type loaded: valid, expired, revoked (via a status list the CS can resolve); query by type scope | Response contains only the valid credential | SHOULD | 5.4.2 | | ✅ | | Tag rule; expired, not-yet-valid and revoked credentials are filtered out. IH's own tests cover a mixed set of one type, of which only the valid credential is presented |
| CS-PRES-13 | Validate a returned VP externally: signature verifies against the holder DID document, referenced key carries the `authentication` relationship, VP holder matches the CS's participant DID | All generated VPs satisfy the verifier validation rules of §5.4.3 | MUST | 5.4.3 | | ✅ | | TCK checks VC contents but not the VP-level `authentication` relationship. Until eclipse-edc/IdentityHub#1063 IH published no verification relationships at all, so no generated VP satisfied §5.4.3; signing keys are now declared for authentication and withdrawn on revocation. The VP's `iss` is the participant DID |
| CS-PRES-13 | Validate a returned VP externally: signature verifies against the holder DID document, referenced key carries the `authentication` relationship, VP holder matches the CS's participant DID | All generated VPs satisfy the verifier validation rules of §5.4.3 | MUST | 5.4.3 | | ✅ | | TCK checks VC contents but not the VP-level `authentication` relationship. IH published no verification relationships at all until #1063, so no generated VP satisfied §5.4.3; signing keys are now declared for authentication and withdrawn on revocation, and the VP's `iss` is the participant DID |

Deferred: all Presentation Definition cases, including the `501 Not Implemented` response (§5.4.1.1) for non-supporting implementations. The three PD tests in the TCK are `@Disabled` upstream (`cs_05_04_01_01_presentationRequest`, `cs_05_04_01_02_lessTypesThanAuthorizedByTypeRequest`, `cs_05_04_01_01_invalidPresentationEscalationRequest`); if enabled upstream, they join the executed set.

Expand All @@ -301,7 +301,7 @@ Interoperability runs pairing implementations in both directions per §1.1: DIV
|---|---|---|---|---|---|---|---|---|
| RT-01 | Request → issue → deliver, once per commonly supported DCP profile | Holder ends up holding a verifiable credential; issuer status reads `ISSUED`; both sides agree on `issuerPid`/`holderPid` correlation | MUST | 6.1, A.2 | `is_6_4_1_credentialRequest` (issuer side, vc11), `cs_06_05_01_credentialMessage` (holder side, vc11) | ✅ | | `vc20-bssl/jwt` never exercised by the TCK; IH's own e2e covers both profiles |
| RT-02 | Offer → request → issue | Same end state, initiated by the issuer's offer | MUST | 6.6.1 | | ✅ | | TCK tests offer reception only; auto-request is an implementation choice; IH's own e2e covers the full flow |
| RT-03 | Rejection round trip: request accepted, then rejected post-acceptance | Issuer status reads `REJECTED`; issuer delivers a `CredentialMessage` with `status: REJECTED`, correct pid correlation, no `credentials` payload required, `rejectionReason` discloses nothing confidential; holder observes the failure; nothing stored on the holder | MUST | 6.5.1, 6.8.1 | | ✅ | | eclipse-edc/IdentityHub#1063 completes both halves: the issuer sends a `CredentialMessage` with `status: REJECTED` and the pids of the failed process when it reaches its terminal error state, and the holder fails the request on receipt. The status-endpoint path (IS-REQ-02, CS-REQ-07) is unchanged |
| RT-03 | Rejection round trip: request accepted, then rejected post-acceptance | Issuer status reads `REJECTED`; issuer delivers a `CredentialMessage` with `status: REJECTED`, correct pid correlation, no `credentials` payload required, `rejectionReason` discloses nothing confidential; holder observes the failure; nothing stored on the holder | MUST | 6.5.1, 6.8.1 | | ✅ | | Both halves are in place: the issuer sends a `CredentialMessage` with `status: REJECTED` and the pids of the failed process when it reaches its terminal error state, and the holder fails the request on receipt. The status-endpoint path (IS-REQ-02, CS-REQ-07) is unchanged (#1063) |
| RT-04 | Batch: one request for multiple credential types | All credentials arrive in one `CredentialMessage` and are individually usable | MUST | 6.5.1 | | ✅ | | is_6_4_1 exercises a 2-credential request but does not assert single-message delivery; IH's own e2e covers the batch round trip, with both credentials stored and individually correct |
| RT-05 | Revocation: (1) issued credential carries a `BitstringStatusListEntry` with resolvable `statusListCredential` URL, valid index, `statusPurpose: revocation`; (2) status list dereferences to a well-formed signed Bitstring Status List credential, bit unset; (3) revoke via management interface; (4) bit now set | End-to-end revocation works and is externally verifiable; an independent verifier rejects the revoked credential | MUST | 6.10 | | ✅ | | TCK tests verifier behavior against a TCK-hosted list only — nothing tests the issuer publishing or flipping status; IH's own tests cover publication, rollover, and the revocation round trip |
| RT-06 | Key rotation: issuer rotates its signing key | New key appears in the issuer DID document while the old `verificationMethod` is retained; credentials issued before rotation remain verifiable until expiry; newly issued credentials verify against the new key | SHOULD | 6.9.1 | | ⚠️ | | §6.9.1's MUST (retention period at least to the last credential's expiry) applies if rotation is supported; IH has an e2e for rotation but it is `@Disabled` and needs its own runtime, because rotating the issuer key mutates state shared by its whole test class |
Expand Down Expand Up @@ -333,7 +333,7 @@ These cases complete the conformance surface beyond the issuance and presentatio

**IH column totals:** 2 ❌, 3 ⚠️ unknown, 1 — (pre-auth n/a), the rest ✅.

Nine cases were closed by [eclipse-edc/IdentityHub#1063](https://github.com/eclipse-edc/IdentityHub/pull/1063) and are green subject to it merging: the `REJECTED` `CredentialMessage` path on both sides (CS-STOR-05, RT-03), the `vc.id` scope alias (CS-PRES-11), `CredentialObject` completeness and id stability (IS-META-02), per-offer reasons (IS-OFF-03), credential filtering and VP validity (CS-PRES-12, CS-PRES-13), and `kid` handling on the holder's endpoints (TOK-09, TOK-11).
Nine cases were closed by [eclipse-edc/IdentityHub#1063](https://github.com/eclipse-edc/IdentityHub/pull/1063), merged 2026-08-29: the `REJECTED` `CredentialMessage` path on both sides (CS-STOR-05, RT-03), the `vc.id` scope alias (CS-PRES-11), `CredentialObject` completeness and id stability (IS-META-02), per-offer reasons (IS-OFF-03), credential filtering and VP validity (CS-PRES-12, CS-PRES-13), and `kid` handling on the holder's endpoints (TOK-09, TOK-11).

Two of those were not gaps this plan had identified. Both are security-relevant and worth noting for the DIV assessment, since a BYOW Credential Service can get them wrong the same way:

Expand Down
Loading