Skip to content

rubrics: reset child rubrics when a rubric is reset - #1535

Open
TheDeadcoder wants to merge 1 commit into
openenv:mainfrom
TheDeadcoder:fix/rubric-reset-children
Open

TheDeadcoder wants to merge 1 commit into
openenv:mainfrom
TheDeadcoder:fix/rubric-reset-children

Conversation

@TheDeadcoder

@TheDeadcoder TheDeadcoder commented Oct 30, 2026 •

Copy link
Copy Markdown

Summary

Rubric.reset() now resets every registered child rubric, so Environment._reset_rubric() and direct rubric.reset() calls clear stateful rubrics (such as TrajectoryRubric) nested in Sequential, Gate, WeightedSum, RubricList and RubricDict. TrajectoryRubric.reset() calls super().reset() to keep the same contract.

Fixes #1533

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • New environment
  • Refactoring

Alignment Checklist

Before submitting, verify:

  • I have read .claude/docs/PRINCIPLES.md and this PR aligns with our principles
  • I have checked .claude/docs/INVARIANTS.md and no invariants are violated
  • I have run /pre-submit-pr (or bash .claude/hooks/lint.sh and tests) and addressed all issues

RFC Status

  • Not required (bug fix, docs, minor refactoring)
  • RFC exists: #___
  • RFC needed (will create before merge)

The change implements the composition already documented in RFC 004. Composites that override reset() (for example REPLRubric) are unchanged.

Test Plan

New regression tests, all failing on main and passing with this change:

  • tests/core/test_rubrics/test_containers.py::TestContainerReset (each container, and a nested composition)
  • tests/core/test_rubrics/test_trajectory_rubric.py::TestTrajectoryRubricBasics::test_reset_resets_child_rubrics
  • tests/core/test_rubrics/test_environment_integration.py::TestEnvironmentRubricLifecycle::test_composed_rubric_resets_children_between_episodes
  • tests/core/test_rubrics/test_async_environment_integration.py::TestAsyncEnvironmentRubricIntegration::test_composed_rubric_resets_children_on_reset_async
PYTHONPATH=src:envs uv run pytest tests/core/test_rubrics -q

Rubric suite: 191 passed. Full CI test command: 4253 passed, 137 skipped; 10 textarena_env tests failed only because NLTK word data could not be downloaded, and they fail the same way on main without that data. usort check, ruff format --check and ruff check pass on the changed files.

Notes:

  • A child instance shared at two places in the tree is reset twice; reset() is idempotent.
  • Children implementing only reset_async() are still reached only as the root rubric (_reset_rubric_async() behavior unchanged).

Claude Code Review

N/A


Note

Low Risk
Targeted lifecycle bug fix in rubric composition with broad regression tests; no auth, I/O, or API surface changes beyond correct reset propagation.

Overview
Rubric.reset() now walks registered children and calls reset() on each, so episode boundaries clear state on nested rubrics (especially TrajectoryRubric) inside Sequential, Gate, WeightedSum, RubricList, and RubricDict. Previously only the root was reset, so trajectories could leak across Environment.reset() / reset_async() when the root was a container.

TrajectoryRubric.reset() calls super().reset() before clearing _trajectory, matching the documented override pattern. Subclasses with custom reset() should still call super().reset() to propagate (unchanged for composites that fully override reset, e.g. REPLRubric).

Regression coverage adds container, trajectory-child, sync/async env integration tests.

Reviewed by Cursor Bugbot for commit 9cb55f4. Bugbot is set up for automated code reviews on this repo. Configure here.

Rubric.reset() now resets every registered child, so stateful rubrics nested in Sequential, Gate, WeightedSum, RubricList and RubricDict (such as TrajectoryRubric) no longer keep the previous episode. TrajectoryRubric.reset() calls super().reset() to keep the same contract.

Signed-off-by: TheDeadcoder <sakibbuet2k19@gmail.com>
@TheDeadcoder
TheDeadcoder force-pushed the fix/rubric-reset-children branch from d926e33 to 9cb55f4 Compare October 30, 2026 10:07

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Composed rubrics keep child state across environment reset

1 participant