feat: actor deregistration with safety checks, CLI, admin UI (#56) - #63
Open
rcbevans wants to merge 20 commits into
Open
feat: actor deregistration with safety checks, CLI, admin UI (#56)#63rcbevans wants to merge 20 commits into
rcbevans wants to merge 20 commits into
Conversation
…sult from public API
Adds GET /actors (lists actor_config rows with active job counts and
schedule counts) and POST /actors/{actor}/deregister (deregister with
force + purge_queue form params, gated on admin_actions_enabled).
- src/taskq/web/admin/actors.py: route module following the workers.py
pattern with CSRF validation and admin_actions_enabled gate
- src/taskq/web/templates/actors.html: table view with per-row
deregister form (force/purge_queue checkboxes, confirm dialog)
- src/taskq/web/templates/_base.html: Actors nav link after Workers
- tests/test_web_admin_actors.py: 4 integration tests covering page
listing, form rendering, 403 on disabled actions, and successful
deregister redirect + row deletion
- pyproject.toml: S608 per-file ignore for the new test file
These modules are shared by client, CLI, admin UI, testing, and __init__ — not worker-internal. Relocating them from taskq.worker.* to taskq.* fixes the layering violation where client code reached into the worker package.
- M1: Admin UI now consumes notice query param and shows success banner - L1: Remove unused structlog loggers from _actors.py and admin/actors.py - Add missing 409 test for deregister with active jobs - Add full-stack integration tests via TaskQ.actors client path
…NG for purge, spec doc fixes
…y, CLI output assertions
…e2e purge assertion, doc gaps
…fixture convergence, template/doc fixes
…tus sets, force-aware error message
…es, schema assertion, edge-case tests L1: Document admin UI limitation for actor names containing '/' L2: Whitelist notice query param to fixed messages (XSS prevention) L4: Fix warning wording — sweep transitions to terminal, not heals L5: Add cron fire/re-enable race warning to docstring L6: Add concurrent deregistration orphaned-queue warning to docstring L7: Type-annotate status_counts as dict[str, int] with explicit casts L10: Document ActorsClient as Postgres-only in class docstring L11: Clarify ActorNotFoundError scope in docstring L12: Distinct CLI exit codes: 2=refusal, 3=not-found (was: 1 for all) L15: Add set_capacity None-vs-UNSET forwarding test L16: Add refusal gap tests: scheduled-only, jobs+schedules precedence, invalid schema L17: Assert WARNING message and exception prefix in CLI tests L18: Add negative CSRF test for deregister route L19: Strengthen property test schema assertion, replace admin_pool with module_pg_pool, tighten e2e >=1 to ==1
rcbevans
force-pushed
the
spec/actor-deregistration
branch
from
July 30, 2026 04:58
57b45a3 to
048da63
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
client.actors.deregister()— a first-class actor deregistration API with defined safety semantics — so ephemeral per-run actor deployments can clean up theiractor_configand orphanedqueuesrows without hand-rolled SQL. Also adds thetaskq actor-config deregisterCLI command and an/admin/actorspage with a deregister button. The default path refuses deregistration while non-terminal jobs or enabled cron schedules reference the actor;force=Truecancels pending/scheduled jobs, disables schedules, and proceeds (running jobs always block).Closes #56.
Implementation
deregister_actor()— ops layer (src/taskq/actor_config_ops.py)Transaction-scoped function (
conn.transaction()) with two modes:force=False(default): refuses if any non-terminal jobs (pending/scheduled/running) reference the actor (ActorHasActiveJobsErrorwith per-status counts), or if any enabled cron schedules reference it (ActorHasEnabledSchedulesErrorwith schedule IDs). If both checks pass, deletes theactor_configrow.force=True: refuses only if running jobs exist (running jobs are actively executing; the dispatch query inner-joinsactor_config, so a running job that retries would be stranded, and the terminal-write path'sCOALESCEforresult_ttlwould silently fall back to the@actor(...)literal). Cancels pending/scheduled jobs (marks ascancelledwitherror_class='ActorDeregistered'), disables enabled cron schedules (setsenabled=false— not delete, so the operator can re-enable if the actor is re-registered), then deletes theactor_configrow.Terminal job history (succeeded/failed/cancelled/crashed/abandoned) is never deleted or modified —
jobs.actoris plaintext, not an FK, so terminal rows remain queryable by actor name after deregistration.DeregisterResult.terminal_jobs_remainingreports the count.purge_queue=Truedeletes the orphanedqueuesrow only when no otheractor_configrow references the same queue name (shared queues are never purged).ActorNotFoundErroris raised if noactor_configrow exists.The TOCTOU race under READ COMMITTED is documented: callers must quiesce the actor (stop enqueuing, disable cron, wait for running jobs to terminate) before calling
deregister.Exception hierarchy (
src/taskq/exceptions.py)ActorsClient— pool-wrapping facade (src/taskq/client/_actors.py)Acquires a connection from the injected
asyncpg.Poolper call, delegates toactor_config_ops. Methods:list(),get(),set_capacity(),deregister().TaskQ.actorsproperty (src/taskq/client/_taskq.py)Returns the
ActorsClientinstance (created duringopen()). RaisesRuntimeErrorif called beforeopen().CLI (
src/taskq/cli.py)Exit code 0 on success (prints
DeregisterResultsummary), 1 on refusal or not found.Admin UI (
src/taskq/web/admin/actors.py,actors.html)GET /admin/actors— lists allactor_configrows with active job counts and enabled schedule countsPOST /admin/actors/{actor}/deregister— deregister withforceandpurge_queueform params; CSRF-protected; gated onadmin_actions_enabled; returns 404 for unknown actor, 409 for refusal, 303 on successPublic API exports (
src/taskq/__init__.py)ActorsClient,DeregisterResult,ActorDeregistrationError,ActorHasActiveJobsError,ActorHasEnabledSchedulesError,ActorNotFoundErrorare importable fromtaskq.Breaking changes
actor_config.pyandactor_config_ops.pymoved fromtaskq.worker.to the top-leveltaskq.package. Update imports:from taskq.actor_config_ops import deregister_actor(wasfrom taskq.worker.actor_config_ops import ...).worker/_transient.pyremoved.TRANSIENT_PG_ERRORSandUnexpectedLoopErrorGuardno longer exist. Call sites inleader.py,_leader_sweeps.py,heartbeat.py,run.pynow inline the specific exception tuples they need.dispatcher_command_timeoutsetting removed fromWorkerSettings. Leader loops no longer wrap iterations inasyncio.timeout(); dedicated connections no longer setcommand_timeout.watchdog_dump_after_fractionsetting removed fromWorkerSettings. Straggler dumps now fire immediately during shutdown countdown.test_watchdog_safety.pydeleted (1969 lines) — tested the removedUnexpectedLoopErrorGuard/dispatcher_command_timeout/dump_after_fractionsurfaces.Test coverage
tests/test_exceptions.pyTaskQError→ActorDeregistrationError→ specific errors)tests/test_actor_deregistration.pyforce=Falserefusal paths (pending/running jobs, enabled schedules),force=True(cancels pending/scheduled, disables schedules, refuses running, keeps terminal history),purge_queue(orphaned, shared, noop when queue row absent), idempotency (double deregister), combined force+purge, concurrent deregistrationtests/test_actors_client.pyActorsClientdelegation wiring (list/get/set_capacity/deregister) with fake pool; error propagationtests/test_actor_deregistration_client.pyTaskQ.actors— deregister clean, not found, active jobs refusal, force cancels, purge queue, double deregister, list, gettests/test_cli_actor_deregister.py--force,--purge-queue, not found exit 1, active jobs error exit 1, schedules error exit 1, output format, double deregistertests/test_web_admin_actors.pytests/test_taskq_actors_property.pytq.actorsraises beforeopen(), returnsActorsClientafteropen()tests/e2e/test_actor_deregistration.pyConcurrency safety
test_concurrent_deregister_one_succeeds_one_raises— two concurrentderegister_actorcalls on the same actor via separate asyncpg connections. Under READ COMMITTED, both pass the safety checks, but only oneDELETE ... RETURNINGreturns a row; the other gets 0 rows and raisesActorNotFoundError. The transaction rolls back cleanly on the losing side.Closes #56