diff --git a/.claude/skills/model-family-onboarding/SKILL.md b/.claude/skills/model-family-onboarding/SKILL.md index e88a1b11..bbaead08 100644 --- a/.claude/skills/model-family-onboarding/SKILL.md +++ b/.claude/skills/model-family-onboarding/SKILL.md @@ -23,7 +23,7 @@ next family gets it in an afternoon rather than a rediscovery. independent sources repeat it *and* nothing primary contradicts it. - **Model knowledge is data, never engine code.** Template JSON, stored prompts, a builtin's system prompt, README prose, a skill. No per-model - Python (proposal `docs/proposals/agent-catalog-legibility.md`, "Principle"). + Python (proposal `docs/proposals/agent-catalog-legibility-complete.md`, "Principle"). - **Do not transcribe a prompt format the vendor publishes.** Point at it. If the vendor ships an agent skill (MiniMax does) or a system-prompt constant inside diffusers (Lightricks does), the dw skill says "use that" and a test diff --git a/.vscode/settings.json b/.vscode/settings.json index 73ac4371..3f3f5857 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -7,6 +7,7 @@ "aiohttp", "controlnet", "cuda", + "denoise", "dtype", "exif", "fullgraph", diff --git a/CLAUDE.md b/CLAUDE.md index 64084f75..c7066661 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -120,7 +120,7 @@ read an inferred workspace back as one the user named - `get_prompt_dir` yields to its older discovery (`./prompts`, then the walk up from the workflow file) for an inferred workspace but not for an explicit one. `--workflow-dir`, `--output-dir` and `--prompt-dir` each still override one folder. See -docs/WORKSPACES.md, and docs/proposals/server-workspaces.md for the later stages +docs/WORKSPACES.md, and docs/proposals/server-workspaces-complete.md for the later stages (workflow search path, run directories, `asset:`/`output:` references). ### Type System @@ -154,6 +154,27 @@ docs/WORKSPACES.md, and docs/proposals/server-workspaces.md for the later stages rooted at the library rather than the workflow file. The library is `DW_PROMPT_DIR` / `--prompt-dir`, else `./prompts` if it exists, else found by walking up from the workflow file's directory +- A step carrying `for_each` (a list, or `variable:` naming one) is expanded by + `expand_for_each` (`dw/for_each.py`) into one ordinary step per entry, named + `@`, immediately after `replace_variables` in + `Workflow.run` and, with the caller's arguments folded, in `validation_errors`. + Inside a member `item:` / `item:field` is the entry (any type, spliced whole); + a later step reads the group with `gather:` (a list; splices inside a + list); two groups over the same list pair by key (`slice` inside `shot@x` is + `slice@x`). `previous_result:` naming a group is a directed error. `@` is + reserved in step names; entry names are validated and unique; 32 entries max; + `release_pipeline`/`release_models` survive on the last member only. The + realized workflow keeps `for_each`; the manifest names the members. An entry + of a list-valued variable may reference another variable + (`"from_file": "variable:character_a_voice"`); `resolve_variable_values` + (`dw/variables.py`) replaces those once, before `realize_args`, refusing a + cycle, and `undeclared_variable_references` walks inside list/dict variable + values too. The catalog derives `lists` (`list_fields`, `dw/for_each.py`): + the fields an entry takes are the `item:` references the steps make, `name` + first; an entry key no step reads is a validation warning + (`entry_field_warnings`). A `cost` entry may carry `per_entry` + (`{variable, minutes, entries}`), measured, never derived. An empty + `for_each` list is an error; `expanded_definition` realizes constants first. - Every run directory holds `workflow.json` beside its manifest: the *realized* workflow, with the run's arguments folded into the variable defaults, the seed it used, stored prompt text inlined and `output:.../latest/...` pinned to the @@ -219,8 +240,28 @@ same reason - default setup cannot load a pack. literal `previous_result:` or `from_previous_result` naming no *earlier* step, with the JSON path it sits at. References otherwise resolve lazily per step, so a step renamed in one place and not another failed only when the run reached it, after - every step before it had generated. A reference spelled by a `variable:` is left - alone - what it names is not knowable before substitution + every step before it had generated. The definition is substituted before the check, + so a reference spelled by a *declared* variable is checked by its value; one spelled + by an undeclared variable is itself a validation error (below) +- **`for_each` expands before the reference check** — `validation_errors` substitutes + (the caller's `arguments` when they are all good, else the defaults) and expands + first, so `gather:` and `item:` errors carry the path of the template step + (`steps[0].for_each[1].name`). Expansion records each expanded step's *source* index + (`expand_for_each(definition, source_indices)`), so a reference error always carries a + path in the file the author wrote, and one inside a member names the member in its + message. An undeclared `variable:` is a validation error at the path it sits at, not a + warning and not a complaint about the `for_each` list that did substitute: once a + `variables` block exists, `replace_variables` refuses an undeclared reference, so it is + a run that cannot start +- **The two MiniMax cut templates take one `shots` list** — since the stage-2 + rewrite (2026-09-11) `templates/minimax/dialogue-short` and `music-video` + have no `shot_N_*` variables; a scripted caller passes `shots` (entries + `{name, prompt, references, num_frames}` and `{name, prompt, start_frame}`). + The members are `shot@` in the manifest and the gallery. This is the + breaking change the next release note should name. The CLI and REPL only + take `name=value` strings, and a string handed to a list variable is + comma-split - so `shots` can only be supplied over the API/MCP (a JSON + body); `python -m dw.run` runs the templates' default list - **Cartesian product explosion** — multiple `previous_result` references multiply: 4 images × 3 masks = 12 iterations - **Component sharing requires exact key matching** between `shared_components` and `reused_components` - **Built-in workflows** need explicit argument mapping: `"prompt": "variable:prompt"` diff --git a/README.md b/README.md index b84de269..ef382219 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,7 @@ once; without it the run fails partway through with a 401/403 from the Hub. ## Drive it from an agent -Then just ask. The agent has 50 tools covering the whole surface — the +Then just ask. The agent has 55 tools covering the whole surface — the workflow catalog, the real diffusers pipeline signatures, the job queue, the gallery, the model cache: diff --git a/docs/MCP.md b/docs/MCP.md index 8b7cc315..68500bd9 100644 --- a/docs/MCP.md +++ b/docs/MCP.md @@ -187,7 +187,7 @@ Nothing in this sequence costs GPU time. ## Tool reference -52 tools in six groups. Names and arguments below are transcribed from +55 tools in six groups. Names and arguments below are transcribed from `dw_mcp/server.py` — nothing here is renamed or reshaped for the docs. ### Catalog (read-only) @@ -212,8 +212,8 @@ when no single workflow covers it. | --- | --- | --- | | `list_guides()` | — | List the documentation the engine serves: each guide's name, what it covers, and its section headings. The index is the routing table - match a request's shape against a heading rather than guessing | | `get_guide(name, section=None)` | `name`, `section` | Get one guide whole, or one section of it. Prefer a section: a guide runs to thousands of lines. Section names match loosely, so a heading copied approximately still resolves | -| `list_workflows(shape=None, traits=None, configures=None, include_models=False)` | `shape`, `traits`, `configures`, `include_models` | List stored workflows. Always the server's compact view: each entry carries `summary`, `shape`, `traits`, `cost`, `kinds`, `variable_names`, and `configures` only when set - `get_workflow` has the full description and definition. `shape` keeps one of `image`, `image-set`, `image-edit`, `shot`, `sequence`, `audio`, `text`, `utility`; `traits` is comma-separated and every one listed must match (`has-audio`, `chained`, `image-conditioned`, `identity-referenced`, `needs-input-media`, `composes-workflows`); an unknown value in either is a 400 listing the vocabulary. Templates only by default - `configures=