diff --git a/docs/MCP.md b/docs/MCP.md index ea3985ab..2597d4b3 100644 --- a/docs/MCP.md +++ b/docs/MCP.md @@ -254,7 +254,7 @@ The session starts in `default` and stays there unless it is told otherwise. | `use_workspace(name)` | `name` | Work in that workspace for the rest of the session - every later call reads and writes there. This is how to keep your work out of another agent's namespace rather than sharing the default one. Checked against the server, so a typo fails here rather than scoping every later call to nothing | | `create_workspace(name, use=False)` | `name`, `use` | Create a workspace. Pass use=true to switch this session to it as well; otherwise the session stays where it was and the result says so | | `delete_workspace(name, acknowledged_cost=False)` | `name`, `acknowledged_cost` | Permanently delete a workspace and everything in it. Refuses without the acknowledgement, reporting what it would remove | -| `list_assets()` | — | The input media on the server, each with the `asset:` reference a workflow argument carries. Look here before asking for a file - what a workflow needs may already be there | +| `list_assets()` | — | The input media on the server, each with the `asset:` reference a workflow argument carries. Look here before asking for a file - what a workflow needs may already be there. `libraries` names the roots searched and which are writable; `shadowed` lists names a nearer library hides | | `keep_output(name, asset_name=None, overwrite=False, shared=False, workspace=None)` | `name`, optional `asset_name`, `overwrite`, `shared`, `workspace` | Keep a generated file as an input asset under a stable `asset:` name, so a later workflow can rely on it. The copy happens on the server: nothing is downloaded or re-uploaded. `asset_name` may name a folder and takes the kept file's extension when it has none; `shared=true` keeps it in the library every workspace shares, which is where a recurring cast belongs. `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it | | `upload_asset(file_path, asset_name=None, shared=False)` | `file_path` | Push a local image, video or audio file into the server's asset library and get back its `asset:` reference. The file is read from the machine the MCP server runs on, so this is how an input reaches a dw.serve running somewhere else. `asset_name` stores it under a readable name (`cast/priya-voice.wav`) instead of a random one; `shared=true` puts it in the library every workspace shares | | `delete_asset(name)` | `name` | Permanently remove one file from the asset library, by the name `list_assets` reports. Deletes from whichever library holds it - this workspace's own before the shared one; one from a read-only examples library is refused. Any workflow still carrying that `asset:` reference stops loading | diff --git a/docs/SERVER.md b/docs/SERVER.md index 346c7335..099d4187 100644 --- a/docs/SERVER.md +++ b/docs/SERVER.md @@ -435,7 +435,12 @@ The editor's forms come from these; they are just as usable from scripts: - `GET /api/assets` — the asset library: input media, each with the `asset:` reference a workflow carries rather than a path, since a path only means something on the server's own machine. Empty rather than an - error when no library is configured + error when no library is configured. `libraries` lists the roots searched, + in order, each `{origin, dir, writable}` — what `asset_dirs` names without + saying which of them an upload or delete can actually reach. `shadowed` + lists the entries a nearer library hides: same shape as an `assets` entry + but without `url` (that URL would serve the shadowing file, not this one), + plus `shadowed_by` naming the origin that won - `POST /api/assets/keep` (`{"name": ..., "asset_name": ..., "overwrite": false, "shared": false}`) — keep a generated file as an input asset under a stable name, returning its `asset:` reference. A run's files are named by the run that made them, @@ -455,6 +460,22 @@ The editor's forms come from these; they are just as usable from scripts: workspace's own before the shared one, the order `asset:` resolves in). An asset from a read-only examples library answers 403, the same as a read-only prompt or workflow; a name nothing holds answers 404 +- `POST /api/assets/archive` — `{"names": [...]}` (1-1000) bundles a + multi-file asset selection into one zip, named by each file's + library-relative path, which is the name its `asset:` reference carries. + The gallery archive's counterpart on the input side; it resolves down the + same search path a run does, so a selection spanning this workspace's + library, the shared one and an examples tree downloads as one archive, and + an unknown or out-of-library name 404s the whole request rather than + yielding a partial one. A duplicate name (repeated in the selection, or + differing only by leading/trailing whitespace) collapses onto the one zip + entry. Media (image/video/audio) stores rather than deflates, unless it's + a raw format that still compresses (`.bmp`, `.wav`) - everything else the + libraries hold is an already-compressed container, and the response does + not start until the archive is complete, so deflating it is latency the + caller waits through for nothing. Everything else - `.json`, `.md`, + `.txt`, an unrecognized extension - deflates; so does the export zip's + text files (`workflow.json`, `manifest.json`, `job.json`, the README) - `POST /api/uploads?filename=...` — the raw bytes of one image, video or audio file (200MB ceiling, checked from `Content-Length` before a byte is read, and again on the body; extension held to the allowed image/video list), saved diff --git a/docs/superpowers/plans/2026-09-14-assets-page-libraries.md b/docs/superpowers/plans/2026-09-14-assets-page-libraries.md new file mode 100644 index 00000000..242da196 --- /dev/null +++ b/docs/superpowers/plans/2026-09-14-assets-page-libraries.md @@ -0,0 +1,244 @@ +# Assets page: libraries as sections, review fixes, and cleanups + +Branch: `assets-page` (over `develop`). Spec: the code review findings in this +session plus the section-by-library design accepted by the user. There is no +separate spec file; the Global Constraints below are the binding text. + +## Global Constraints + +- Python: never `eval`/`exec`/`shell=True`; every disk read of a request-named + path goes through `validate_path(path, base)` with a non-None base (see + CLAUDE.md *Security Rules* - CodeQL models the validators as barriers). +- Tests: Python `python -m pytest tests/test_server.py tests/test_server_downloads.py tests/test_mcp_assets.py -q` + (run the focused file while iterating, the three once before committing); + UI `cd ui && npm test` and `npm run check` must both pass before a commit + that touches `ui/`. +- Every UI string a test asserts is quoted in the task; use it verbatim. +- Existing response fields of `GET /api/assets` (`asset_dir`, `asset_dirs`, + `assets`, `folders`, and every field of an `assets` entry) keep their names + and meaning - MCP `list_assets` returns this body verbatim. +- Commit per task, message in the repo's voice (imperative subject, a body + saying *why*), ending with `Co-Authored-By: Claude Fable 5.1 `. +- Comments explain *why*, in the style of the surrounding code; no narration + of what the line does. +- Do not touch `ui/src/lib/pages/GalleryPage.svelte` beyond what a task names. + +## Task 1: Archive routes - one search probe, one compression policy, one tail + +Files: `dw/server/app.py`, `tests/test_server_downloads.py`, `tests/test_server.py`, +`docs/SERVER.md`, `dw_mcp/CLAUDE.md`. + +1. **`_asset_in(name, roots)`** (app.py ~2262): stop calling + `resolve_asset_reference` per root (each call already searches the pinned + fallbacks, so the loop re-walks them - measured 107 syscalls per hit vs + 29). New body: `validate_asset_reference(name)` once (import from + `dw.assets` if not already; it raises `SecurityError` subclasses), then for + each root `candidate = os.path.join(root, name)`; if + `os.path.isfile(candidate)` return `validate_path(candidate, root)`. On a + total miss raise `HTTPException(404, detail=f"Unknown asset {name!r}: not found in {', '.join(roots)}")`; + when `roots` is empty the detail is + `f"Unknown asset {name!r}: this workspace has no asset library"`. A + `SecurityError` from validation becomes a 404 with `str(e)` as today. + Keep the docstring's *why* (bulk callers resolve many names against one + path). Fix the comment in `archive_assets` (~3075) so it claims only what + the code does. +2. **`_asset_file(reference, ws)`** stays a one-line wrapper; add a module-level + helper inside `create_app` named `_resolution_roots(ws)` returning + `_asset_roots(ws) or [ws.assets]` with a docstring saying why a missing + library is still named (so the 404 points at the caller's own directory), + and use it at the three sites (validate route ~1498, `_asset_file`, + `archive_assets`). +3. **`archive_assets`**: strip each name and dedupe preserving order before + resolving (`names = list(dict.fromkeys(n.strip() for n in body.names))`); + the zip entry name is the stripped name. Test: a body + `["iris.png", "iris.png "]` yields one entry named `iris.png`. +4. **Compression policy**: replace `COMPRESSIBLE_EXTENSIONS` with + `RAW_MEDIA_EXTENSIONS = {".bmp", ".wav"}` defined directly beneath + `MEDIA_KINDS` (~2237) with a comment: the allowlist members that are not + already-compressed containers. In `_zip_download`, an entry is + `ZIP_STORED` only when `extension in MEDIA_KINDS and extension not in RAW_MEDIA_EXTENSIONS`; + everything else (`.json`, `.md`, `.txt`, `.bmp`, `.wav`, unknown) is + `ZIP_DEFLATED`. Tests: (a) `assert RAW_MEDIA_EXTENSIONS <= set(MEDIA_KINDS)` + - expose both via `app.state` or import; pick whatever the existing test + for `MEDIA_KINDS` (if any) does, else attach them to `app.state` -; (b) the + export zip's `workflow.json` entry has `compress_type == zipfile.ZIP_DEFLATED` + (extend an existing export test in `tests/test_server_exports.py` or + `test_server_downloads.py`); (c) the existing + `test_archive_stores_already_compressed_media_rather_than_deflating_it` + still passes. +5. **One tail**: add `_archive_selection(entries, kind)` beside + `_zip_download`: it builds the zip via `_zip_download(entries, f"dw-{kind}s-{stamp}.zip")` + with `stamp = datetime.now().strftime("%Y%m%d-%H%M%S")`, and logs + `f"Archived {len(entries)} {kind} files"` *after* the archive is written. + Both archive routes call it (`kind` = `"output"` / `"asset"`), and the + duplicated three-line tails go. Check the download filename the UI/tests + expect - keep whatever `dw-outputs-*.zip` / `dw-assets-*.zip` names exist + today. +6. **Export route** (~3424): restore a plain two-line loop naming `path` and + `entry` (`entry = os.path.relpath(path, current).replace(os.sep, "/")`, + append `(f"{job_id}/{entry}", path)`), and restore the dropped "not a + second permanent copy" comment if git shows it (`git show develop:dw/server/app.py` + around the old export route). +7. **Docs**: `docs/SERVER.md` `POST /api/assets/archive` paragraph - replace + the "stored rather than deflated unless `.bmp`, `.wav`" sentence with the + new rule (media stored, everything else deflated; the export zip's text + files deflate). `dw_mcp/CLAUDE.md` line 10: "the gallery's bulk zip" -> + "the two bulk zips (gallery and assets)". + +## Task 2: `GET /api/assets` names its libraries and what they shadow + +Files: `dw/server/app.py` (`list_assets` ~2906), `tests/test_server.py`, +`docs/SERVER.md`, `docs/MCP.md` (the `list_assets` row), `dw_mcp/assets.py` +docstring, `ui/src/lib/types.ts`. + +1. Response gains `libraries`: a list in search-path order, one per root + `_asset_roots(ws)` returns, `{"origin": <"workspace"|"common"|"examples">, "dir": , "writable": }`. + `asset_dirs` stays (it is `[lib["dir"] for lib in libraries]`). +2. Response gains `shadowed`: entries the loop currently skips at + `if relative in seen: continue`. Each has the same fields as an `assets` + entry **except `url`** (the URL would serve the shadowing file) plus + `"shadowed_by": `. + `assets` is unchanged - exactly the files `asset:` resolves to. + Record the shadowing origin as you go (a dict `name -> origin` filled when + a name is first seen). +3. Tests in `tests/test_server.py` beside `test_the_asset_library_lists_what_it_holds`: + (a) `libraries` lists the workspace root first with `writable: true`; an + examples root (see `test_gallery_metadata_finds_an_asset_an_examples_tree_brought` + for how one is configured) is `writable: false`; (b) a name present in + both the workspace and an examples library appears once in `assets` + (origin `workspace`) and once in `shadowed` with `shadowed_by: "workspace"` + and no `url` key; (c) with no library configured the body has + `libraries: []` and `shadowed: []`. +4. `ui/src/lib/types.ts`: extend `AssetListing`'s type (find where + `listAssets` return type is declared - `api.ts` or `types.ts`) with + `libraries: AssetLibrary[]` and `shadowed: ShadowedAsset[]`; + `AssetLibrary = { origin: AssetFile['origin']; dir: string; writable: boolean }`, + `ShadowedAsset = Omit & { shadowed_by: AssetFile['origin'] }`. + Update `AssetsPage.test.ts`'s `listing` fixture to include + `libraries: [{ origin: 'workspace', dir: '/ws/assets', writable: true }]` + and `shadowed: []` so `npm run check` passes; do not change the page. +5. Docs: `docs/SERVER.md` `GET /api/assets` bullet gains one sentence each + for `libraries` and `shadowed` (why: a client that only sees the + resolved list cannot show which of its names hide a shared one). + `docs/MCP.md` `list_assets` row and `dw_mcp/assets.py` docstring mention + that `shadowed` lists names a nearer library hides. + +## Task 3: A bulk bar counts what its actions will touch + +Files: `ui/src/lib/picks.svelte.ts`, `ui/src/lib/picks.test.ts`, +`ui/src/lib/BulkBar.svelte`, `ui/src/lib/pages/AssetsPage.svelte`, +`ui/src/lib/pages/GalleryPage.svelte`, their tests, `ui/CLAUDE.md`. + +1. `Picks.size` returns `this.names.length` (ticks the filter hides are + inert: not counted, not acted on, kept until the filter lifts). Delete + `get hidden()` and its test. Update the doc comment on `size`. +2. `keepFailed(attempted: string[], failed: string[])`: delete every name in + `attempted` that is not in `failed`; touch nothing else (a hidden tick + survives). Update both callers (`removePicked` in each page passes + `names, failed`). +3. `picks.test.ts`: replace the `hidden` test with (a) "size counts only what + the order includes" - tick two, shrink the order to one, `size === 1`, + restore the order, `size === 2`; (b) "keepFailed keeps ticks the action + never attempted". +4. `AssetsPage.svelte` / `GalleryPage.svelte`: `downloadPicked` / + `removePicked` early-return when `picks.names.length === 0` (the bar is + hidden then anyway, but a keyboard path could still reach them). +5. `ui/CLAUDE.md` lines ~113-116: rewrite the `Picks.hidden` / open-question + sentences to state the rule now chosen: the count is the visible + selection, a hidden tick waits. +6. Run `npm test` and `npm run check`; fix any test that asserted the old + count. + +## Task 4: The assets page sections by library + +Files: `ui/src/lib/pages/AssetsPage.svelte`, `ui/src/lib/pages/AssetsPage.test.ts`, +`ui/CLAUDE.md`, `ui/src/lib/api.ts` only if `listAssets` needs its type. + +Depends on Task 2's `libraries`/`shadowed` fields and Task 3's `Picks`. + +Structure - the search path is the top level, folders inside: + +1. Derive `sections` from `libraries` in order: for each library, + `{ origin, dir, writable, label, assets: visible.filter(a => a.origin === origin), shadowed: listing.shadowed.filter(s => s.origin === origin && matches filter) }`. + Labels (verbatim): `workspace` -> `This workspace`, `common` -> + `Shared library`, `examples` -> `Examples`. A library with no assets and + no shadowed entries still renders its header (so an empty workspace shows + where an upload would land) unless a filter is active, in which case an + empty section is skipped. +2. Each section: a header row with the label, `(N)` count, the `dir` in + `.path` mono, a collapse chevron (state in `storageGet/storageSet` under + key `collapsed-asset-libraries`, a `Record`; while the + filter is active every section is open, as `FolderGroups` does), and: + - `writable` and origin `workspace`: an `Upload` button (the one filled + button on the page). + - `writable` and origin `common`: a `.quiet` `Upload to shared` button with + `title="lands in the shared library - visible from every workspace under this root and cannot be moved afterwards"`. + - not writable: a muted `read-only` span. + One hidden ``; a `let uploadTarget: 'workspace' | 'shared'` + set by whichever button was clicked before `fileInput.click()`. The + `window.prompt` text keeps its shape: `Upload — name in the ${uploadTarget} asset library:`. +3. Inside an open section: `FolderGroups` with `collapseKey="collapsed-asset-folders-${origin}"`, + the existing card snippet. Then, if the section has shadowed entries, one + more grid titled with a muted `shadowed/` heading rendered the way + `FolderGroups` renders a folder name (reuse its markup style, not the + component): tiles with class `cell shadowed` (opacity 0.45, no checkbox, + not a button - a `div`), caption = leaf name, `title="shadowed by this workspace's {name} - asset:{name} resolves to that file"` + where the shadowing origin's label is used in place of "this workspace's" + when `shadowed_by !== 'workspace'` (`shadowed by the shared library's ...`). +4. Remove: the `origin` state and ` - {#if originOffered} - - {/if} - - + An asset is input a workflow names by reference: an argument set to asset:name - loads this file at run time, whatever run produced it. A name in this - workspace shadows the same name in a shared or example library. + loads this file at run time, whatever run produced it. The shared library and + any examples library sit on every workspace's search path, so they follow you + between workspaces; only this workspace's own section changes with the picker, + and a name here hides the same name further down. {#if loaded && !error && assets.length === 0} @@ -190,44 +295,142 @@ {/if} - a.name)} - groupOf={(name) => folderByName.get(name) ?? ''} - collapseKey="collapsed-asset-folders" + - {#snippet card(name)} - {@const asset = byName.get(name)!} -
+ {busy} + noun="asset" + ondownload={downloadPicked} + ondelete={removePicked} +/> + +{#each sections as section (section.origin)} +
+ + {section.dir} + + {#if !section.writable} + read-only + {:else if section.origin === 'common'} + + {:else if section.origin === 'workspace'} - {#if asset.origin !== 'workspace'} - {asset.origin} + + {#if isOpen(section.origin)} + a.name)} + groupOf={(name) => folderByName.get(name) ?? ''} + collapseKey="collapsed-asset-folders-{section.origin}" + {filterActive} + minColumn="150px" + > + {#snippet card(name)} + {@const asset = byName.get(name)!} +
+ + {#if section.writable} + picks.toggle(name, e.shiftKey)} + /> + {/if} + +
+ {/snippet} + + + {#if section.shadowed.length} + +
+ shadowed/ ({section.shadowed.length}) - {/if} -
- {/snippet} - +
+
+ {#each section.shadowed as entry (entry.name)} +
+ +
+ {entry.kind} + {leaf(entry.name)} +
+
+ {/each} +
+ {/if} + {/if} +{/each} -{#if loaded && assets.length > 0 && visible.length === 0} +{#if loaded && assets.length > 0 && ordered.length === 0}

Nothing matches "{filter}".

{/if} @@ -247,7 +450,9 @@ title="open the file itself in a new tab">open file {selected.kind} · {kb(selected.size)} · {day(selected.mtime)}{selected.kind} · {formatBytes(selected.size)} · {formatMtime( + selected.mtime, + )} {#if selected.origin === 'examples'} {/if} -{#if assetDirs.length} -

- read from - {#each assetDirs as dir, index (dir)} - {dir}{#if index < assetDirs.length - 1}, - {/if} - {/each} - {#if assetDir} - · uploads land in {assetDir} - {/if} -

-{/if} -