[PR]: Add Test Specification

[xdxxxdx]

The previous PR used the imec account, which triggered a policy warning requiring its removal. As a result, this PR has been reintroduced for our testing specification. Please feel free to provide feedback. Apologies for any inconvenience.
The purpose of this PR is to introduce a basic set of testing specifications for implementing a TREE client.

[pietercolpaert]

I’ve made an AI overview of the current spec tests which I think is a great start:

The document is a useful test-plan skeleton, but it is not yet a reliable client conformance specification. Several tests are too weak to verify the named feature, while others require behavior that TREE does not currently specify.

Main findings

1. The relation tests produce false positives

All nine comparator tests use one relation and require the client to retrieve every member. A client that does not understand the comparator but simply follows every tree:node link will pass them all.

That behavior is already required for unknown or unpruned relations: the client must follow the link unless it can safely prune it ([01-tree-specification.bs](/home/pieter/Projects/treecgspecification/01-tree-specification.bs)).

A real comparator test needs:

• A client query or region of interest.
• At least one relevant and one irrelevant branch.
• An observable request log.
• An assertion that the relevant branch was followed.
• An assertion that the irrelevant branch was pruned.
• Verification that the returned members are complete for the query.

2. Some tests impose requirements absent from TREE

TREE does not specify:

• A particular initialization error message.
• Automatic network retries.
• A five-second performance threshold.
• General behavior for malformed relations.
• A client API exposing “initialization complete” or “synchronization complete.”

Those can be implementation-quality tests, but not TREE conformance criteria.

3. Most member-extraction requirements are untested

The suite only counts member identifiers. It does not test whether the correct quads are extracted. Missing cases include:

• Focus-node subject quads.
• Recursive blank-node inclusion.
• Named graphs named after the member.
• Dereferencing an out-of-band member.
• Open and closed SHACL shapes.
• Required and optional paths.
• Nested sh:node shapes.
• sh:and, sh:or, and sh:xone.
• Avoiding quads belonging to another member.

These constitute a substantial part of client behavior in the TREE and Shape Topologies specifications.

4. The tests mix producer and client conformance

Statements such as “all members in the target node start with abc” test whether the TREE publisher constructed a valid relation. They do not test whether a client correctly interprets that relation.

Publisher fixtures should first be declared valid. Client assertions should then concern requests, pruning, extracted members, and metadata.

5. There is a specification inconsistency around substring relations

The test describes tree:SubstringRelation using SPARQL CONTAINS, which is semantically reasonable. However, the vocabulary currently says it must conform to SUBSTR ([vocabulary.md](/home/pieter/Projects/treecgspecification/vocabulary.md)). That specification issue should be resolved before making the test normative.

Test-by-test assessment

Test	Assessment	Reason
1.1.1	Partially correct	Valid scenario, but “initializes without errors” does not prove that the collection and root node were identified.
1.1.2	Mostly correct, too weak	The expected result correctly mentions the redirected IRI and tree:view, but the pass criterion only checks successful initialization.
1.1.3	Incorrect as conformance	TREE does not require a particular error or even define “invalid tree:Node” sufficiently for this test.
1.2.1	Partially correct	Complete traversal is appropriate, but comparing only counts can hide missing, duplicate, or incorrectly extracted members.
1.3.1–1.3.9	Misclassified	They test ordinary link traversal, not comparator understanding or pruning.
1.4.1	Correct with clarification	Empty nodes are explicitly allowed. The fixture should clarify whether the empty node has further relations.
1.4.2	Reasonable SHOULD-level test	TREE says clients should detect faulty trees. A circular structure is invalid publisher data, so this is robustness rather than ordinary conformance.
1.4.3	Incorrect	A string is not inherently invalid for a comparator. More importantly, silently skipping the target link can lose members; an unsupported relation should normally be followed, not pruned.
1.5.1	Not a conformance test	The five-second threshold is arbitrary and environment-dependent.
1.5.2	Not measurable	“Without memory leaks or degradation” has no workload, duration, baseline, or threshold.
1.5.3	Not a TREE requirement	Retry policy and network-error reporting are not specified by TREE.

Important missing core tests

The most important additions would be:

• Initialization from a collection IRI, including dereferencing its unique tree:view.
• Redirects while following tree:node, not only during initialization.
• Extraction of all relation descriptions from every visited node.
• Following unknown relation types without pruning.
• Combining multiple relations to the same node using logical AND.
• SHACL property paths, including alternative and inverse paths.
• Multiple path results interpreted as logical OR.
• Unicode normalization and case-sensitive comparison.
• IRI ordering and xsd:dateTime without timezone.
• tree:GeospatiallyContainsRelation.
• Duplicate members appearing in multiple nodes.
• Combining multiple tree:viewDescription resources.
• Search forms.
• Separate optional conformance classes for discovery, shape topology, and the TREE profile.

Recommendation

I would describe the current document as an initial test plan, not yet as a conformance specification. A stronger structure would define explicit conformance classes:

• Core initialization and traversal
• Member extraction
• Individual relation comparators
• Shape topology
• Search forms
• Discovery
• Streaming/profile support

Each normative test should provide concrete RDF and HTTP fixtures, identify the exact specification requirement being tested, and assert semantic results or observable requests—not merely “no error.”

The Bikeshed document itself builds successfully. I made no file changes.