Skip to content

[Bug] Skill lifecycle hooks missing from the trace mappings, acs.md, and the Trace pillar docs #58

Closed

Description

@rocklambros

The three skill lifecycle hooks (skillRegister, skillLoad, skillUnload) are present in hooks.md, in the specification.md taxonomy table, and as schemas under specification/v0.1.0/hooks/. They are absent from six other surfaces that restate the hook set, two of which are normative.

Found while reviewing #20, whose FAQ correctly states 19. The suite is green on all of this (198 passed).

Normative gaps

specification/v0.1.0/trace/otel-mapping.json and specification/v0.1.0/trace/ocsf-mapping.json each enumerate 16 steps/* keys and contain none of the three skill hooks.

$ python3 -c "import re;print(len(set(re.findall(r'\"(steps/[A-Za-z]+)\"', open('specification/v0.1.0/trace/otel-mapping.json').read()))))"
16

Both files apply to ACS-Trace, and otel-mapping.json states that "v0.1.0 conformance requires that a deployment emitting OTel for the Trace pillar uses these span names and required attributes verbatim." conformance.md:19 says the skill hooks SHOULD be implemented. So a deployment that implements skillRegister and claims ACS-Trace has no normative span name or class_uid to emit. Two conformant implementations can emit incompatible telemetry for the same hook and nothing adjudicates.

The skillRegister case is the one worth caring about, because a denied skillRegister is a malicious skill definition blocked at the vetting gate, which is exactly the event a SIEM should receive.

Documentation gaps

  • docs/acs.md:11 says "16 native lifecycle hooks" and enumerates 16 names. grep -c "skillRegister\|skillLoad\|skillUnload" docs/acs.md returns 0. This is the ACS overview page, so it is where a reader learns what ACS-Core is, and it currently tells them skill hooks are not part of v0.1.0.
  • docs/spec/trace/events.md has zero occurrences of "skill". Its OTel span table and OCSF class table both stop at subagentStart/subagentStop.
  • docs/spec/trace/extend_ocsf.md has zero occurrences of "skill".

hooks.md:3 says 19. So the published site currently states both 16 and 19 in prominent places.

Why CI does not catch it

tests/test_hook_taxonomy.py derives the hook set from specification/v0.1.0/hooks/*.json and compares it against two files:

SPECIFICATION = ROOT / "docs" / "spec" / "instrument" / "specification.md"
HOOKS_PAGE = ROOT / "docs" / "spec" / "instrument" / "hooks.md"

Those are the two files that are already correct. The guard's premise is that the hook set is stated in three places. It is stated in at least nine, and four of the unguarded ones are wrong today.

Suggested fix

  1. Add the three hooks to otel-mapping.json and ocsf-mapping.json with span names and class_uids consistent with the existing 16.
  2. Update docs/acs.md:11, docs/spec/trace/events.md, and docs/spec/trace/extend_ocsf.md.
  3. Widen the guard so it derives the restating surfaces rather than hardcoding two paths. Matching every steps/<name> occurrence across docs/ and specification/v0.1.0/trace/ would have caught all six.

Item 3 matters more than the other two. Without it the next hook addition reopens the same gap, which is what happened here.

Activity

  1. sabasiddique1 commented on Sep 9, 2026

    @sabasiddique1
    Contributor

    Picking this up. I reproduced all six gaps on a fresh clone: both trace mappings hold 16 steps/* keys, acs.md:11 / events.md / extend_ocsf.md have no skill rows, and test_hook_taxonomy.py only guards the two surfaces that are already right.

    I've got a fix working locally, test first. The widened guard now discovers every surface that restates the hook set under docs/ and specification/v0.1.0/trace/:

    • exact keys on both mapping JSONs (steps/ prefix only, so agbom/* stays out)
    • a restating-surface check
    • a count check on every "N native hooks" claim

    On the unfixed tree it goes red on all six. With the fix, the three skill hooks are in otel-mapping.json and ocsf-mapping.json (span names and required attributes taken from each hook's own required fields), and acs.md, trace/events.md, trace/extend_ocsf.md are updated. Suite is green. Strict MkDocs build passes.

    One extra thing the guard caught that isn't in the list above: extend_ocsf.md also omits steps/agentTrigger. Included it; I'll call it out separately in the PR.

    The OCSF class choice for the skill hooks is a judgment call, so I'm flagging it as a reviewer decision in the PR rather than deciding alone.

    Opening the PR shortly, would appreciate a look when it's up.

  2. moved this to Needs triage in ACS Project Trackeron Sep 10, 2026
  3. added
    type:bugSomething isn't working
    scope:in-focusFeeds the ninety-day committed outcome. Maintainers only
    on Sep 10, 2026
  4. moved this to Needs triage in ACS Project Trackeron Sep 10, 2026
  5. changed the title [-]Skill lifecycle hooks missing from the trace mappings, acs.md, and the Trace pillar docs[/-] [+][Bug] Skill lifecycle hooks missing from the trace mappings, acs.md, and the Trace pillar docs[/+] on Sep 10, 2026
  6. added a commit that references this issue on Oct 13, 2026
    185ef4e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority:P2Maintainers onlyscope:in-focusFeeds the ninety-day committed outcome. Maintainers onlystatus:acceptedIn the backlog. Maintainers onlytype:bugSomething isn't working

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions