Add library job tracking and status monitoring - #37
Conversation
Surface running workflow state back to admins. Cloudflare Workflows (library reload / add-document) were fire-and-forget with no visibility. Backend: - New `library_jobs` D1 table (+ migration) recording each workflow run: type, status, label, triggeredBy, per-group result, timestamps. - Trigger routes create a `running` row and pass a jobId into the workflow; the workflow records its own completion. - Workflows now propagate per-group error messages instead of swallowing them, and derive a `partial` status when some groups fail but others succeed. - Editor-gated `GET /api/library-jobs/library/:libraryId` lists a library's jobs and reconciles stale `running` rows against the live Cloudflare instance status. - Access level is cached in KV (~1h) so frequent polling doesn't hit Onshape on every request. Frontend: - `useLibraryJobsQuery` polls while a job runs (aggressive then backing off), only for editors. - A watcher auto-refreshes the library and toasts the outcome when a job finishes; a navbar spinner shows while one is running. - New `/app/library-jobs` page lists recent jobs with status, duration, and per-group failure detail, linked from admin settings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Follow-up on review feedback. The first cut rendered per-group load results on the jobs page, which duplicated the existing build-status system (buildIssues + LOAD_FAILED/INSERTABLES_FAILED, shown inline per entity). This moves the per-entity signal onto the entities and slims the run record. - Add lastLoadedAt (time of last successful load) to groups and insertables; stamp it in saveGroup/saveInsertable. Failures leave the last-good time untouched and stay expressed by build checks, so there's no separate per-entity load-status column. - Surface it in the build-status hover card as a dedicated "Loaded 3h ago" header pinned to the top-right (not another row); "Never loaded" when null. - Drop the per-group result column from library_jobs; it's now a slim run-log for the running indicator, auto-refresh, and a compact recent-reloads list (status/label/who/when/duration). The workflow still derives overall complete/partial/errored status. - Regenerate a single clean 0001 migration (creates library_jobs without result, adds last_loaded_at to groups/insertables). Verified additive against origin/cert, which only has 0000. - Extract formatRelativeTime/formatDuration into a shared helper; add load-status and build-status test coverage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Remove the library_jobs table, its routes/helpers/DTO, the jobs page, navbar indicator, watcher, and workflow job-recording — the workflows go back to fire-and-forget. What's kept is the useful part: each group and insertable stores its last successful load time (lastLoadedAt), surfaced in the build-status hover card. To keep the UI from spinning forever, add a small bounded poll: after a reload or add-document is triggered, watch the library's cacheVersion (bumped by the workflow when it finishes) and refresh the library view once it changes, then stop (with a hard time cap). No DB or job records involved. The KV access-level cache stays (it's a general per-request win and keeps the cacheVersion poll cheap). Migration regenerated to a single additive 0001 that only adds last_loaded_at to groups and insertables; verified against origin/cert, which has only 0000. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Refactor the editor-only build-status hover card so the admin flags are editable in place, instead of living as separate toggle items in the right-click "Admin options" submenu. - The hover card now shows: last-loaded time, Build checks, an Admin section of Switches, and (insertables) a read-only Parsed section. - Insertable switches: Visible to users, Insert and fasten, Part number search, and Open composite (part studios only). Group switch: Sort alphabetically. Each is disabled while its mutation is in flight; the hide toggle keeps its favorites-removal confirm. - Parsed (read-only) shows Vendors and Configurable — Configurable stays automatic, not a toggle. - Bumped the HoverCard closeDelay so the card can be moved into and its switches clicked. - Extracted the toggle mutations into reusable hooks in card-hooks.ts. - The admin context menu keeps the commands: reload thumbnail (both), and for groups show/hide all, reorder, delete, add group. The per-flag toggle menu items were removed since they're switches now. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Mantine's notifications.show is a no-op when a toast with the same id is already displayed (notifications.store: returns early on duplicate id), so the "loading spinner → replace with same id" pattern used by add-group, insert, and the error handlers left the loading toast stuck open forever — e.g. the reload/add "Adding document…" toast never closed. Route the show* helpers through a showToast that first hides any existing toast with the given id, so a loading toast is reliably replaced by its final info/success/error state. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
The admin switches felt frozen: toggling bumped the library cacheVersion, which changed the build-status query key, blanking the hover card (data went undefined, the badge returned null and the card closed) while the switch waited on a sometimes-slow Onshape request. - Toggle mutations now optimistically flip the build-status cache in onMutate (mirroring useSetGroupOrderMutation), show a loading→success/ error toast, and revert on failure — so switches respond instantly. Factored into a shared useOptimisticToggleMutation; visibility gets the same treatment. - getBuildStatusQuery uses keepPreviousData so the card no longer blanks out and closes while the next cacheVersion refetches. - Split the badge into BuildStatusCard (hover content wrapping the menu) and BuildStatusBadge, which takes a semantic `hoverMenu` prop instead of children. - Dropped the disabled-while-pending state on switches (optimistic updates make it unnecessary). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Rework the admin hover card around a proper header and clearer sections: - Header: the group/insertable name in bold, a severity summary of pill badges (e.g. "1 error", "1 warning", or a green "All checks pass"), and the last-loaded time on the right. - Build checks: each issue rendered as a tinted callout box in its severity color; the section is hidden entirely when everything passes. - Admin: each toggle now has a short description under its label. - Parsed: unchanged (Vendors, Configurable), under an uppercase header. Threads the entity name into InsertableStatusBadge/GroupStatusBadge (no thumbnail, per the earlier decision). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
The open-composite and sort-order toggle toasts reported the opposite action (turning "open composite" on said "Removed open composite"). Both were <void> mutations whose messages branched on the captured current value, which the optimistic update flips before onSuccess fires. Convert them to <boolean> mutations that receive the target value, matching insert-and-fasten and part-number-search. Also: "info" instead of "note" for the info count badge, nudge the issue callout icon down to align with the first text line, drop trailing periods from build-check messages, and rename the header time label to "changed". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
- Move CLAUDE.md to AGENTS.md and add a comment-brevity guideline; CLAUDE.md now just imports AGENTS.md. - access-level-cache: drop the defensive KV try/catch and shorten the doc comments to a single sentence each. - services/load-insertable/workflows: trim or delete narrating comments. - build-status: carry elementType on InsertableBuildStatus so the badge reads it from the query instead of a threaded prop; move getIssueMessage into build-checker as getIssueDescription so build-issue definitions live in one file. - card-hooks: replace the monolithic useOptimisticToggleMutation config with a small patchBuildStatus helper and explicit per-toggle hooks; drop the loading toast for the trivial (DB-only) open-composite and sort-order toggles; standardize error messages on "Unexpectedly failed to update X". - theme: set cursorType "pointer" globally instead of a per-switch style. - reload-refresh: take the baseline cacheVersion from the caller's loader data instead of reading the query cache; tighten comments. - build-status.test: name the seeded last-loaded constants. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
- Add a small p-limit (createLimiter, LOAD_CONCURRENCY=15) threaded through
LoadContext so a whole-library reload loads at most 15 insertables at once.
This keeps Onshape pressure down; because each insertable's save is its own
durable step, a rate-limit burst only affects the in-flight few and earlier
work is preserved.
- Track the active reload workflow instance per library in KV (reload-lock).
/reload-groups now skips creating a second reload while one is live and
returns { status: "already-running" }; add-group jobs stay unrestricted. The
reload button shows an info toast instead of starting a second poll. KV has no
CAS, so a sub-second double-trigger can still race — acceptable for an
admin-only action.
- Raise limits.subrequests to 200,000 in the cert and production env blocks so a
large reload doesn't hit the per-invocation subrequest cap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Hiding pops a confirm modal, but Mantine's HoverCard only closes on mouse-leave — which never fires when the modal overlay merely covers the dropdown, so the card was left stranded behind the modal. Expose a close function via context (implemented by remounting the uncontrolled HoverCard) and call it from the visibility switch before the hide modal opens. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Editors can trigger a reload that runs for minutes with no persistent signal. Expose the per-library reload state (isReloadRunning) via a polled GET /job-status endpoint and show a spinner while it runs — in the navbar (editors only) and in the build-status hover card, where it replaces the last-modified line. Both are tooltipped "A job is running". The reload trigger and completion invalidate the job-status query so the spinner appears/clears promptly rather than waiting for the 10s poll. Also rename the card's "Changed" label to "Last modified" and give it a clock icon. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
… into claude/workflow-state-ui-qme9hy
- Align the header's "Last modified" line with the title by putting them on a shared centered row (badges move below). - Add a tooltip clarifying "Last modified" is the last time changes were pulled from Onshape (distinct from a reload merely checking the doc). - Size the navbar spinner to match the other navbar icons (18px). - Turn each row's build-status severity icon into a spinner while a job runs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Replace the frontend reload-refresh polling globals with a hook that refreshes the library when the polled job status flips from running to idle (useRefreshLibraryOnJobFinish, mounted once in the editor-gated navbar indicator). Removes the redundant reload loading/success toasts — the spinner now signals a running load. Generalize the reload-lock into a job tracker (renamed job-tracker.ts) that records both reload and add-group workflow instances per library, so /job-status reports any running load job (isAnyJobRunning) and add-group keeps auto-refreshing when it finishes. Reloads stay a singleton via isReloadRunning. Bump the tracked-job KV TTL to 24h since a reload can run for hours. Also: reword the spinner tooltip to say the library is loading from Onshape in the background; tidy the concurrency limiter (no p-limit jargon, clearer queue drain); trim the reload-singleton comment. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Each workflow now removes its own tracked-job entry in a final untrack-job step (it knows event.instanceId), so the running-job state clears promptly instead of lingering until the TTL. Only its own entry is touched, so concurrent jobs are unaffected; the status-check in activeJobs stays as the backstop for a job that crashes before untracking, and the 24h TTL is now purely that crash safety net. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Show a transient "started" toast when a reload or add-document trigger call returns, and a success toast when the job-status poll sees the job finish (alongside the library refresh). No persistent running toast — the spinner covers that. Drop the now-unused hideToast helper. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
The "refetch context-data then re-run the loaders" dance was duplicated across the job-finish watcher and the card admin mutations, and the "invalidate favorites then re-run loaders" dance across the three favorites mutations. Pull them into useRefreshLibrary / useRefreshFavorites and use them everywhere those patterns appeared. useRefreshLibrary also drops the now-redundant explicit library invalidate: every library-mutating endpoint bumps cacheVersion, which re-keys the version-keyed library/search/build-status queries, so refetching context + re-running loaders is enough to pull fresh data. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Group delete cascade-removes insertables and their favorites, and reorder bumps the version, so both now use useRefreshLibrary (which pulls the new version's library/search/build-status and re-prefetches favorites) instead of a bare library invalidate. Reorder keeps its optimistic update and its error-path rollback. Also extract useUpdateContextData (optimistic context patch + loader re-run) and use it for the settings save and the access-level switch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Extract a refetchFavorites helper and call it directly from useRefreshLibrary (and useRefreshFavorites) so a library refresh explicitly refetches favorites — a group delete or element hide cascades into them — rather than relying on the /app loader's incidental favorites prefetch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Reordering bumps the library version, so onSettled's useRefreshLibrary reconciles the optimistic order on success. Remove the special-case error invalidate (and the now-unused libraryQueryMatchKey import). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
Mirror the card-toggle pattern: capture the previous library data in onMutate (and cancel in-flight fetches) and restore it in onError, so a failed reorder reverts immediately instead of relying on a version bump that never happens on error. onSettled still refreshes on success. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
- Fold the KV access-level cache into access-level-utils.ts (with the rest of the access-level code); delete access-level-cache.ts. - Rename load/concurrency.ts -> load/load-common.ts. - Collapse the job-status query + refresh-on-finish into a single useJobStatus hook in refresh.ts; delete job-refresh.ts and drop the completion toast (we don't reflect the outcome). The navbar just calls useJobStatus(). - Drop the manual optimistic-undo in onError across the card mutations and the group reorder; useRefreshLibrary now invalidates the library and build-status queries so a failed mutation rolls back via refetch (these routes aren't CDN-cached, so a same-version refetch returns fresh data). onError is now just the error handler; the onSettled var is named refreshLibrary. - build-status: CountBadge takes a severity and maps to color/noun itself; drop the uppercase section headers for normal capitalization. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
AlexKempen
left a comment
There was a problem hiding this comment.
Left comments
| } | ||
|
|
||
| /** | ||
| * Refreshes the whole library view: refetch the context (its cacheVersion keys |
There was a problem hiding this comment.
Oh hell naw this comment should be two lines max
| } | ||
|
|
||
| /** | ||
| * Polls whether a load job is running and refreshes the library once it |
There was a problem hiding this comment.
Just the first sentence
| export function useJobStatus(): boolean { | ||
| const refreshLibrary = useRefreshLibrary(); | ||
| const running = useJobStatusQuery().data?.running ?? false; | ||
| const wasRunning = useRef(running); |
There was a problem hiding this comment.
Let's use a useState instead of a ref for this I think.
If there's a reason we need a ref specifically, add a comment inline explaining why
| ) => void { | ||
| const router = useRouter(); | ||
| return useCallback( | ||
| (recipe: (data: ContextData) => void) => { |
There was a problem hiding this comment.
Make a custom type for the callback function.
Give it a better name than recipe, like updateFunction or something.
| const jobRunning = useJobStatus(); | ||
| if (!jobRunning) return null; | ||
| return ( | ||
| <Tooltip label="The library is being loaded from Onshape in the background"> |
| } | ||
|
|
||
| /** | ||
| * Shows a toast. When an `id` is given, any existing toast with that id is |
There was a problem hiding this comment.
REally clean this up, move the comment about existing toasts to the if statement in-line
| "add-group" | ||
| ); | ||
| onSuccess: () => { | ||
| // Replace the loading toast with a transient "started" toast; the |
There was a problem hiding this comment.
Delete this comment
| // Raise the per-invocation subrequest cap so a full library reload | ||
| // (many Onshape calls per insertable) doesn't hit the default limit. | ||
| "limits": { | ||
| "subrequests": 200000 |
There was a problem hiding this comment.
Actually make this 500
| "ADMIN_TEAM": "6a62e6efcc21741bea57362c", | ||
| "NODE_ENV": "development" | ||
| }, | ||
| // Raise the per-invocation subrequest cap so a full library reload |
There was a problem hiding this comment.
Delete this comment
| "ADMIN_TEAM": "5b620150b2190f0fca90ec10", | ||
| "NODE_ENV": "production" | ||
| }, | ||
| // Raise the per-invocation subrequest cap so a full library reload |
There was a problem hiding this comment.
Delete this comment
- refresh.ts: shorten the useRefreshLibrary/useJobStatus doc comments; keep the finished-transition ref (set-state-in-effect is disallowed) with an inline note; give useUpdateContextData a named ContextDataUpdate callback type. - navbar: add withArrow to the running-job tooltip. - notifications: slim showToast's doc and move the Mantine no-op note inline. - card-hooks/add-group: drop now-superfluous comments. - wrangler: drop the subrequest-limit comments and raise it to 500000. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh
…frcdesign/FRCDesignApp into claude/workflow-state-ui-qme9hy
PR #37 added `limits: { subrequests: 500000 }` to the cert and production environments. Wrangler limits are only supported on the Standard Usage Model; on this Free-plan account the deploy is rejected at the versions endpoint with Cloudflare error 100328 ("CPU limits are not supported for the Free plan"), so the post-merge cert deploy failed even though CI (type-check/lint/test) passed. Comment out both limits blocks to restore deploys, keeping them documented so they can be re-enabled after moving to a paid plan. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BSxhQes1omyhxKXe9uHY4K
…#39) PR #37 added `limits: { subrequests: 500000 }` to the cert and production environments. Wrangler limits are only supported on the Standard Usage Model; on this Free-plan account the deploy is rejected at the versions endpoint with Cloudflare error 100328 ("CPU limits are not supported for the Free plan"), so the post-merge cert deploy failed even though CI (type-check/lint/test) passed. Comment out both limits blocks to restore deploys, keeping them documented so they can be re-enabled after moving to a paid plan. Claude-Session: https://claude.ai/code/session_01BSxhQes1omyhxKXe9uHY4K Co-authored-by: Claude <noreply@anthropic.com>
Summary
Adds a new library jobs system to track background workflow executions (document reloads and group additions) with real-time status monitoring. Users with editor access can now view job history and outcomes via a new
/app/library-jobspage, and the UI automatically refreshes when jobs complete.Key Changes
Database & Schema
library_jobstable to track workflow runs with status (running/complete/partial/errored), timestamps, and error messageslastLoadedAtcolumn togroupandinsertablestables to record when entities were last successfully loadedBackend Workflow Integration
LoadLibraryWorkflowto accept alibraryJobIdparameter and report job completion statusfinishLibraryJob()to mark jobs as finished with their outcome (complete/partial/errored)deriveLoadStatus()to compute overall job status from per-group resultsNew Backend Services
library-jobs.ts: Core job management (create, finish, list, reconcile)access-level-cache.ts: KV-backed caching of access levels to reduce Onshape API calls during pollingGET /api/library-jobs/library/:libraryId: Endpoint to fetch recent jobs with editor-only accesslibrary-jobs.test.ts&build-status.test.ts: Test coverage for new endpointsFrontend UI
/app/library-jobsroute displaying a table of recent jobs with status, type, trigger, start time, and durationuseLibraryJobWatcherhook that polls jobs and auto-refreshes library data when jobs finishformat-time.tsutilities for relative time and duration formattingRoute Generation
routeTree.gen.tsto include the new library-jobs route with proper formattingImplementation Details
https://claude.ai/code/session_01LaqfcR5s9LGTXj6AKof7rh