[matrix72c]

Problem

Resource-owning AgentLoops have no framework shutdown hook. When training finishes or raises, a loop can keep background work and external clients alive after the trainer has stopped. In the disaggregated trainer, a producer blocked on generation never observes the shutdown flag, so fit() can hang before any owner is closed.

Change

• Add async AgentLoop.close(), a no-op by default. Both RL trainers close their training and evaluation managers when fit() exits, including on error. Managers forward close() to local loops, Ray actors and router workers; every loop is closed even if another one fails. Close failures are logged and do not replace the training result.
• The disaggregated trainer cancels a producer that is still blocked on generation before closing loop owners. An idle producer still exits through the shutdown flag.
• The built-in sandbox loop rejects new runs, cancels active runners and waits for their existing finally blocks to release their pools.
• Add AgentLoop.prepare_pause() and AgentLoop.resume(), both no-ops by default. Managers call prepare_pause() before pause_generation() and resume() after continue_generation(), in both the colocated and disaggregated managers. Loops whose model calls come from an external harness use them to drain in-flight calls before the abort. If prepare_pause() raises, the manager does not abort inference.

The contract is documented in docs/en/rl/agent_loop_shutdown.md. Existing loops, Runner, Judger and AgentInSandboxLoopConfig usage are unchanged.

Test plan

python -m pytest -q \
  tests/rl/test_agent_loop_shutdown.py \
  tests/rl/test_rl_colocate_trainer.py \
  tests/rl/test_rl_disaggregated_trainer.py \
  tests/rl/test_multi_task_agent_loop_manager.py \
  tests/rl/test_producer.py \
  tests/rl/test_trajectory_logging.py

86 passed on top of current upstream/main. New tests cover normal/error/cancelled trainer exits, training and evaluation owners, local/actor/router dispatch, failure isolation, the sandbox Runner/Pool cleanup path, producer shutdown ordering, and pause/resume hook ordering. Ruff, docformatter and git diff --check pass; mypy reports no errors on changed lines. Tests use fake backends and do not exercise live sandbox deletion or GPU training.