Develop

.github/ISSUE_TEMPLATE/mcp-ticket.md

@@ -0,0 +1,23 @@
+---
+name: MCP agent-loop ticket
+about: Filed by the implementer/tester agent loop (see the iterate repo's CLAUDE.md) — not for human bug reports, use "Bug report" for those.
+title: ""
+labels: ["owner:implementer"]
+---
+
+**tool/endpoint:** which MCP tool or method this concerns
+
+**repro:** exact call made (tool name + params, or command run)
+
+**expected:**
+
+**actual:**
+
+<!--
+Status is tracked by label (status:fixed-pending-verify, status:needs-info,
+status:needs-approval, status:verified) plus the issue's open/closed state
+and close reason (completed = verified, not planned = wontfix/duplicate).
+Owner is tracked by label (owner:implementer, owner:tester, owner:don) —
+exactly one at a time, swapped by whoever hands the ticket off.
+Notes and verify-notes are ordinary comments on the thread, in order.
+-->

CLAUDE.md

@@ -154,6 +154,10 @@ docs/WORKSPACES.md, and docs/proposals/server-workspaces-complete.md for the lat
   rooted at the library rather than the workflow file. The library is `DW_PROMPT_DIR` /
   `--prompt-dir`, else `./prompts` if it exists, else found by walking up from the
   workflow file's directory
+- A step's `result.subfolder` names a subfolder of the run directory for that step's
+  files - by convention `final` for the deliverable and `intermediate` for the rest; any
+  relative path (`shots/act-1`); `variable:`/`item:` allowed; no default. Mechanics under
+  *Result subfolders* in Critical Gotchas
 - A step carrying `for_each` (a list, or `variable:` naming one) is expanded by
   `expand_for_each` (`dw/for_each.py`) into one ordinary step per entry, named
   `<step>@<entry name or index>`, immediately after `replace_variables` in
@@ -298,6 +302,36 @@ same reason - default setup cannot load a pack.
   `JobManager.realized` finds the file. `exports` is a reserved workspace name:
   `POST /api/jobs/{id}/export` gathers one finished job into
   `<workspace>/exports/<job id>/` and `GET /exports/<job id>.zip` streams it.
+- **Result subfolders**: a step's `result.subfolder` (`dw/subfolders.py`) puts its files
+  in a subfolder of the run directory - `<run>/final/x.mp4` - by convention `final` or
+  `intermediate`; the engine treats no name specially and there is no default.
+  `Workflow.step_output_dir` computes the directory once and hands it to both
+  `Result.save` and the pipeline wrapper, so a chain's `save_segments` spill follows it.
+  Shape is `SUBFOLDER_PATTERN` (the `output:` segment rule, so a subfolder is
+  `output:`-addressable up to `OUTPUT_REFERENCE_PATTERN`'s seven-segment ceiling),
+  checked by `subfolder_errors` in `validation_errors` after
+  `for_each` expansion and again at run time; containment is `validate_output_path`
+  against the run directory. Manifest entries and `step_end` carry `subfolder`.
+  `split_run_path` finds the run id anywhere in a path, so `strip_run_id` still groups a
+  workflow's runs. Gallery entries carry it too; `GET /api/gallery?subfolder=` and MCP
+  `list_gallery(subfolder=)` filter on it. The web UI reads the field only:
+  the gallery page offers a subfolder pick once any entry has one, and the
+  job page sections results under `final/` / `intermediate/` headings (or
+  whatever the step named) (`sectionBySubfolder`, `ui/src/lib/results.ts`),
+  unchanged for a run that chose none. `file_base_name` may not contain a
+  separator -
+  it is a name, not a path.
+  Every `workflows/templates/**` file with two or more saving steps
+  marks each one `final`/`intermediate` (`tests/test_template_subfolders.py` pins the rule;
+  `dw/workflows/` builtins stay unmarked - a role is the parent's to assign). That moved
+  the templates' outputs into `<run>/final/` and `<run>/intermediate/`: an
+  `output:<template>/latest/x` reference keeps resolving but stops advancing past the last
+  pre-change run (`keep_output` is the stable form), and a seeded template's first run
+  after the change regenerates rather than hitting the step cache (the key includes
+  `result`). Those two, and a stray `subfolder` key becoming live, are the release-note
+  items beside the `shots` list change. Gallery names for a template's runs now read
+  `<template>/<run id>/final/<file>`, so an `output:` reference built from one carries the
+  `final/` segment
 - **Step cache**: a process-wide singleton (`dw/step_cache.py`) consulted by every `Workflow.run`, including server jobs; entries are keyed by `(workflow id, step name)` and validated against the output
   *root*, never the per-run directory - a run directory is new every execution and would
   defeat the cache; disabled entirely when the workflow sets no `seed`; a hit reports the earlier run's files with `reused: true` and writes nothing new; `memory clear` drops it. This is why "Run again" on a seeded workflow finishes instantly and generates nothing - the job page says so when every step was reused, and `POST /api/jobs/{id}/rerun` with `{"new_seed": true}` (MCP `rerun_job(new_seed=True)`) draws a fresh seed into the workflow's seed variable, which is the way to get a different image

docs/MCP.md

@@ -222,11 +222,11 @@ when no single workflow covers it.
 | `list_tasks()` | — | List every task command a workflow's task step can name |
 | `get_task(command)` | `command` | Get a task command's argument schema |
 | `list_models()` | — | List what the Hugging Face model cache holds, largest first |
-| `get_memory()` | — | Get the worker's VRAM and RAM statistics |
+| `get_memory()` | — | Get the worker's VRAM and RAM statistics. `gpu_*` is the card; `host_memory_rss_mb` / `host_memory_peak_rss_mb` are the worker process's resident and high-water host memory, beside the machine's `host_memory_total_mb` / `host_memory_available_mb` - read both, since an offloading workflow keeps its weights in host RAM and the card says little about what it holds (a host field is absent, not null, where the platform cannot measure it). `live: true` means `info` was measured now and is the worker's own memory; only live readings are comparable with each other. `live: false` with `info: null` (and `stale: false`) means nothing has been measured because nothing is resident; `live: false` with a populated `info` is a cached earlier reading, with `reason` (`job_running`, `worker_stopped`, `worker_busy`, `worker_unreachable`) and `age_seconds` - one cached while a job loads a model understates what is resident, so ask again when the server is idle rather than comparing it with a live figure |
 | `get_health()` | — | Check that the server is alive, and which machine answered: `version`, `device`, whether the worker process is up, the job running now and the queue depth |
 | `get_server_info()` | — | What this installation can do and where it keeps things: `device` (the accelerator a run will use), `version`, the `workspace` this session is working in and the workflow/asset/output/prompt `directories` of *that* workspace, the bind address and port, whether a token is required, and whether MCP is mounted. Check the device before authoring - a CUDA-only choice (bitsandbytes, `torch.compile`, flash attention) is not available on an `mps` or `cpu` server |
 | `list_jobs(limit=20, status=None, workspace=None)` | optional `limit` (newest N), `status` (one state or a comma-separated set of `queued`, `running`, `succeeded`, `failed`, `cancelled`), `workspace` | List queued, running and recent jobs, **newest first**. Bounded by default: the unbounded listing was over a client's tool-result limit on a server with a few months of history, which made it a tool that could not be called at all. `total` says how many matched and `truncated`/`next` say so when the answer was cut - raise `limit` or narrow with `status`. Without `workspace`, a named workspace lists its own jobs and the default one lists every job the server holds |
-| `list_gallery(limit=50)` | `limit` | List generated output files, newest first. A name is `<workflow>/<run id>/<file>`; each entry also carries a ready-made `url`, already scoped to the workspace that made it - a hand-built `/outputs/<name>` URL 404s for anything but the default workspace |
+| `list_gallery(limit=50, subfolder=None)` | `limit`, `subfolder` | List generated output files, newest first. A name is `<workflow>/<run id>/<file>`, where `<file>` may sit in the subfolder the step chose (`final/episode.mp4`); each entry carries `folder` (the workflow) and `subfolder` (by convention `final` or `intermediate`, `''` when the step chose none, any path the workflow wrote otherwise), and `subfolder="final"` lists only deliverables. Each entry also carries a ready-made `url`, already scoped to the workspace that made it - a hand-built `/outputs/<name>` URL 404s for anything but the default workspace |
 | `get_gallery_metadata(name, envelope=False)` | `name` | Get the metadata embedded in a generated file: the exact workflow and arguments that produced it, and, for audio/video, a `media` block (duration, rate, channels, fps, size, peak/mean dBFS). `envelope=true` adds `media.envelope` — `rms_dbfs` and `peak_dbfs` one entry per second — which is what locates something in a track rather than measuring the whole of it |
 
 ### Media
@@ -283,7 +283,7 @@ references written in the same session.
 | Tool | Arguments | Purpose |
 | --- | --- | --- |
 | `run_workflow(workflow_path=None, inline_workflow=None, arguments=None, acknowledged_cost=False, workspace=None)` | exactly one of `workflow_path` (a catalog name from `list_workflows`, with or without `.json`, or a path to a workflow file on the server) or `inline_workflow`, optional `arguments`, `acknowledged_cost`, `workspace` | Queue a workflow for generation. Returns as soon as the job is queued. `workspace` names the workspace for this one call without switching the session to it - use it to pin a job whose `output:` or `asset:` references live in a workspace other than the session's |
-| `get_job(job_id)` | `job_id` | Get a job's status, warnings, output manifest, error and traceback. A running job also carries `progress` (below) |
+| `get_job(job_id)` | `job_id` | Get a job's status, warnings, output manifest, error and traceback; each manifest entry's `subfolder` is the in-run subfolder the step declared - by convention `final` for the deliverable, `intermediate` for scratch, `''` for none. A running job also carries `progress` (below) |
 | `get_job_workflow(job_id)` | `job_id` | The workflow the job actually ran. `realized: true` means every mutable input is pinned (arguments, seed, prompts, `output:latest`); `false` means the job predates run tracking and this is the definition as submitted. Pass it to `save_workflow` to keep it under a name |
 | `export_job(job_id, overwrite=False)` | `job_id`, `overwrite` | Gather one finished job into `<workspace>/exports/<job id>/` on the server: the realized workflow, the run's manifest, the job row, a README, and copies of the assets, earlier-run inputs and outputs. Returns the directory, a zip URL, the file list with sizes and the total. The three JSON files are in the zip, not repeated here - get_job_workflow and get_job serve them individually. **The directory is on the machine running the server**, like `download_output`'s destination - fetch the zip URL and unpack it into `exports/` under the session's working directory (a deliverable, not a temp file); the archive already unpacks into one folder named after the job id |
 | `get_job_events(job_id, after=-1, limit=200)` | `job_id`, `after`, `limit` | Get a page of a job's progress events |

docs/SERVER.md

@@ -54,6 +54,9 @@ load entirely.
   progress, per-step denoising ticks, what each step is doing when it is not
   denoising (loading a model, decoding, saving), and its result files as
   they land.
+  A run whose steps chose a `result.subfolder` shows its results under
+  `final/` and `intermediate/` headings (or whatever the step named), the
+  deliverable first; one that chose none shows them as before.
   Jobs can be cancelled mid-denoise and re-run with one click. A finished
   job's **Export** button gathers the run into the workspace's `exports/`
   on the server (`POST /api/jobs/{id}/export`) and downloads it as one zip -
@@ -78,7 +81,10 @@ load entirely.
   the form view.
 - **Gallery** — everything in the selected workspace's output directory,
   which the engine lays out as `<workflow>/<run id>/`. The folder filter groups a workflow's runs
-  together rather than listing each run separately, and each run directory
+  together rather than listing each run separately, and a **subfolder** pick
+  beside the text filter - offered once any output landed in one - narrows
+  the grid to `final/`, `intermediate/` or whatever a step's
+  `result.subfolder` named; each run directory
   also holds a `manifest.json` describing what produced it (see
   [Workspaces](WORKSPACES.md#runs)). Images generated with
   `embed_metadata` carry their full workflow definition and seed; **open as
@@ -138,7 +144,7 @@ from another machine:
 | --- | --- |
 | `POST /api/jobs` | Queue a run: `{"workflow_path": ...}` or an inline `{"workflow": {...}, "base_dir": ...}`, plus `arguments` for variable overrides. `workflow_path` accepts a stored workflow name as listed by `/api/workflows` (with or without `.json`, nested names included), or a relative/absolute path that still resolves under `--workflow-dir` - confined the same way the `/api/workflows` CRUD routes are; a path that names a real file outside that directory is rejected with 400, not opened. Answers with argument warnings from signature checking. |
 | `GET /api/jobs?workspace=&status=&limit=` | Queue + history summaries, oldest first, with `total` beside them. `status` narrows to one state or a comma-separated set (`queued`, `running`, `succeeded`, `failed`, `cancelled`; anything else is a 400); `limit` keeps the newest N, and `total` still reports how many matched, so a bounded answer cannot be mistaken for a complete one. No parameters means every job, which is what the web UI polls |
-| `GET /api/jobs/{id}` | Full detail: spec, events, manifest, error. A manifest entry for a step served from the step cache carries `reused: true`. A `for_each` step appears in the manifest as its members (`shot@wide_open`, `shot@closeup`), because the manifest records what ran; the run's `workflow.json` keeps the `for_each` form, because it records what was asked |
+| `GET /api/jobs/{id}` | Full detail: spec, events, manifest, error. A manifest entry for a step served from the step cache carries `reused: true`. Every entry carries `subfolder` - the in-run subfolder the step's `result.subfolder` chose, `''` for none. A `for_each` step appears in the manifest as its members (`shot@wide_open`, `shot@closeup`), because the manifest records what ran; the run's `workflow.json` keeps the `for_each` form, because it records what was asked |
 | `GET /api/jobs/{id}/workflow` | The workflow the job ran: `{id, definition, realized, seed_variable}`. `seed_variable` names the variable a `new_seed` rerun would draw into (null when the workflow has none), read from the workflow as written rather than the realized copy, whose seed is pinned. `realized: true` is the copy the run itself wrote (`workflow.json` in its run directory), with arguments, seed, prompts and `output:latest` pinned; `false` falls back to the submitted definition, which is what a job from before run tracking has. 404 means neither is readable - the job itself still is |
 | `POST /api/jobs/{id}/export?workspace=&overwrite=` | Gather one finished job into `<workspace>/exports/<job id>/`: `workflow.json`, `manifest.json`, `job.json`, `README.md`, `assets/`, `inputs/`, `outputs/`. 201 with the file list, total bytes, anything it could not find, a `zip_url`, and the three JSON files inline. 404 unknown job, 409 for a job still running or an existing export without `overwrite` |
 | `GET /exports/{id}.zip?workspace=` | The same tree as one archive, built on request rather than kept as a second copy. Entries are named `<job id>/<relative path>`. Ungated exactly as `/outputs` is |
@@ -162,10 +168,11 @@ Every event in the stream carries a `seq` and an `event` name:
 | `memory` | device memory after a run | `info` |
 | `run_start` | the run directory is chosen, before the first step | `run_id`, `identity`, `run_dir` |
 | `workflow_start` | the run begins | `workflow`, `total_steps`, `steps`, `seed` |
-| `step_start` / `step_end` | each step | `step`, `index`, `total_steps`; `files` at the end. A step served from the step cache adds `reused: true` to `step_end`, and its `files` are the earlier run's files rather than newly written ones |
+| `step_start` / `step_end` | each step | `step`, `index`, `total_steps`; `files` and `subfolder` at the end. A step served from the step cache adds `reused: true` to `step_end`, and its `files` are the earlier run's files rather than newly written ones |
 | `iteration_start` | each argument combination in a step | `step`, `iteration`, `total_iterations` |
 | `pipeline_step` | each denoise step | `step`, `total_steps`. Emitted for a pipeline that takes a `callback_on_step_end`, and for a `ModularPipeline` (H3, LTX-2, Qwen-Image), which takes none - there the denoise block's own progress bar is what reports |
 | `phase` | the step changes what it is doing | `phase`, `detail` |
+| `pipeline_released` | a step with `release_pipeline` drops its pipeline | `step`, `index`, `gpu_memory_allocated_mb` and `gpu_memory_allocated_before_mb` (both `null` where the backend cannot say). Emitted between the step's generation and its files being written, which is where the release happens - so the ordering is readable off the event stream rather than by trying to poll memory through a sub-second write |
 | `workflow_end` | the run finishes | `manifest` |
 
 A step spends most of its wall clock outside the denoise loop, and
@@ -295,7 +302,13 @@ The editor's forms come from these; they are just as usable from scripts:
   queue an enhancement as an ordinary job whose saved text file is the
   result
 - `GET /api/gallery`, `GET /api/gallery/{name}/metadata`,
-  `DELETE /api/gallery/{name}` — outputs and their embedded metadata
+  `DELETE /api/gallery/{name}` — outputs and their embedded metadata. Each
+  gallery entry carries `folder` (the workflow identity, the run id dropped)
+  and `subfolder` (what followed the run id - the `final`/`intermediate` a
+  step's `result.subfolder` chose, `''` when it chose none); `?folder=` and
+  `?subfolder=` filter independently, and the reply's `folders` and
+  `subfolders` list every distinct value over the whole tree, `''` always a
+  member of each so root-level files stay selectable
 - `GET /api/gallery/{name:path}/download` — download an output file
 - `POST /api/gallery/archive` — `{"names": [...]}` (1-1000) bundles a
   multi-file selection into one zip, named by each file's gallery-relative
@@ -376,8 +389,13 @@ The editor's forms come from these; they are just as usable from scripts:
   started) alongside the live `version`/`commit`, so a revert has a
   concrete before/after to compare
 - `GET /api/memory`, `GET /api/health` — worker VRAM/RAM stats and liveness;
-  health also reports `hostname`, `device` and whether `mcp` is mounted, so a
-  remote client can tell which machine answered
+  memory answers `live` (whether `info` was measured by this call), `stale`,
+  `reason` (`job_running`, `worker_stopped`, `worker_busy`,
+  `worker_unreachable`) and `age_seconds`, so a cached reading is never
+  mistaken for the worker's memory now - `info: null` means nothing has been
+  measured because nothing is resident. health also reports `hostname`,
+  `device` and whether `mcp` is mounted, so a remote client can tell which
+  machine answered
 - `GET /api/server` — connection details for the Server page: `hostname`,
   `version`, `device`, the `bind_host`/`port`/`wildcard_bind` the server was
   started with, `auth_required` (whether a token is configured - never the