Skip to content

feat: canonical flow text and per-node notes - #52

Merged
wavekat-eason merged 6 commits into
docs/canonical-documentfrom
feat/canonical-document
Oct 17, 2026
Merged

wavekat-eason merged 6 commits into
docs/canonical-documentfrom
feat/canonical-document

Conversation

@wavekat-eason

@wavekat-eason wavekat-eason commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This implements docs/01-canonical-document.md (#51; this PR is stacked on it). The flow document is the whole truth, so any tool, in either language and offline, edits by parse → change the model → serialize.

Schema

  • An optional note on the flow (after version) and on every node kind (after kind), in v1 and v2. This is a compatible addition: nodes already allow unknown fields, the engine ignores note, and no frozen case changes.

Rust (wavekat-flow)

  • Ordered maps: every generated map is now an indexmap::IndexMap (typify with_map_type), so node, exit and option order survive. serde_json has preserve_order for ui. The crate re-exports indexmap.
  • Flow::to_yaml() writes the canonical text, using a hand-written emitter guided by the bundled schema: property order, required fields and defaults all come from the schema.
  • edit::load(src) / edit::load_draft(src, Missing { id, name }):
    • refuse unknown fields (a schema-driven walk; ui stays opaque) and unsupported schema_versions;
    • return the YAML comments a save drops;
    • load_draft fills a new draft's missing id/name first.
  • Helpers: is_generated_clip_ref (twin of isGeneratedClipRef), is_local_audio_ref, prompt_fields(kind) / PROMPT_FIELDS, and Node::prompt_slots() / prompt_slots_mut() (each prompt with its field name, so a tool can voice or swap prompts in place).

TS (@wavekat/flow-schema)

  • serializeFlow(flow): the twin emitter, guided by the same schema.
  • parseFlow: decodes note, and reports each comment as a comment_ignored warning, found with the yaml library's own lexer. findComments(source) is exported, and its range is a true source offset.
  • mutate.ts: same API, rebuilt on the model.
    • New EditErrors: invalid_document (doesn't decode; validation-only problems are still editable), unknown_field and unsupported_version.
    • stampIdentity sets id/name/version on a draft that lacks them, then goes through the model like every other op.

Fixed in review

  • Comment ranges drifted. findComments counted the lexer's control tokens, so range (and the comment_ignored range) was off by one per scalar before it. It now skips them. A test checks source.slice(range) starts with #.
  • Rust comment finder: it now skips a leading BOM and finds "x"# c, as the TS side does. Lines already agreed.
  • Colon in flow style: Ext 5:30 was written plain inside { ... }, and libyaml (the Rust reader) refuses that. Both emitters now quote : in flow style. The doc rule and the scalars case are updated.
  • Version first: mutate.ts now answers unsupported_version before unknown_field, like edit::load.
  • Tests added: hours unknown fields in edit::load, comment edge cases, version precedence.
  • Checked, no change needed: I diffed the two emitters on 18,000 random documents (odd strings, numbers, integer keys, ui). They were byte-identical, and each text re-read to itself.

Known gap

  • A number of 2^64 or more in ui (for example 1e20) is written in digits, as JavaScript does. The Rust reader can't read an integer that big back. Nothing in the format uses one, so I left it.

Corpus

  • conformance/canonical/: 5 cases (defaults, order, scalars, ui, comments). Both languages must write exactly the canonical bytes, reach a fixed point, and agree on the comment count.
  • Lossless round trip: both languages check that serializing every valid corpus document loses nothing.

Breaking API (0.0.x)

  • Flow::to_yaml() now returns String, not Result.
  • Generated maps and Node::exits() are IndexMap, not HashMap. get/iter/values work the same.
  • Nodes gain a note: Option<String> field. Code that builds Node values directly adds note: None.
  • mutate.ts output drops comments, and its ops refuse documents that don't decode.

This stays feat:, not feat!. We're pre-1.0, and the CLI and voice PRs pin crate 0.0.11 while the platform pins npm ^0.0.14. With bump-minor-pre-major, a feat! would make release-please jump to 0.1.0 and break those pins. As feat: it gives crate 0.0.10 → 0.0.11 and npm 0.0.13 → 0.0.14.

Consumers

These are the draft PRs against this branch, listed in the doc's Consumers section:

  • the platform (adopts the new package, comment-to-note migration, editor note field);
  • the CLI (flow edits through the crate);
  • the voice daemon (crate upgrade).

Test plan

  • cargo fmt --check, cargo clippy --all-targets -D warnings, cargo test -p wavekat-flow (77 unit + corpus), cargo package.
  • pnpm typecheck, vitest (188 tests), pnpm build.
  • Canonical corpus: Rust and TS produce identical bytes; comment counts agree (5 on the tricky case: # inside quotes, block scalars and C# are not comments).
  • TS mutate suite updated for the intended changes. New tests cover the refusals and stamping a draft that has no id.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GNFHhzw4ZsqCFnv6ERigjd

wavekat-eason and others added 2 commits October 17, 2026 22:19
The document is now the whole truth, so editing is parse, change the
model, serialize, in either language (docs/01-canonical-document.md).

- Schema: optional `note` on the flow and every node kind (v1, v2).
- Rust: maps are `IndexMap` (author order kept); `Flow::to_yaml` writes
  the canonical text; `edit::load` / `load_draft` refuse unknown fields
  and unsupported versions and return the comments a save drops;
  `is_generated_clip_ref`, `is_local_audio_ref`, `prompt_fields`.
- TS: `serializeFlow`; `note` decoded; comments reported as
  `comment_ignored`; `mutate.ts` ops rebuilt on the model, refusing
  `invalid_document`, `unknown_field` and `unsupported_version`.
- `conformance/canonical/`: both languages write the same bytes.

API changes: `Flow::to_yaml` returns `String`; generated maps and
`Node::exits()` are `IndexMap`; nodes gain `note`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GNFHhzw4ZsqCFnv6ERigjd
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GNFHhzw4ZsqCFnv6ERigjd
@wavekat-eason
wavekat-eason added this pull request to stack #53 October 17, 2026 11:19
libyaml refuses a ':' inside a plain scalar in flow style, so the Rust reader could not load a canonical text that had one (for example 'Ext 5:30' in options). Both emitters now quote it. The scalars case pins it.
findComments counted the yaml lexer's control tokens in its offsets, so ranges drifted by one per scalar. It now skips them. Rust comments() now skips a leading BOM and finds a comment right after a closing quote, as the TS side does. Adds a range test, edge-case tests and unknown-field tests for hours.
A newer document has fields this build does not know. Rust edit::load already answers with the version error first; mutate.ts now does too.
@wavekat-eason
wavekat-eason marked this pull request as ready for review October 17, 2026 22:38
@wavekat-eason
wavekat-eason merged commit 3abc522 into main Oct 17, 2026
3 checks passed
@wavekat-eason
wavekat-eason deleted the feat/canonical-document branch October 17, 2026 22:41
@github-actions github-actions Bot mentioned this pull request Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant