Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Evaluation results lose the evidence needed to explain retries and gaps
Saved scores alone cannot show which checkpoint and settings were consumed,
whether an execution failed after completing some episodes, or which intended
trials were never launched. Retrying an evaluation must not create an extra
scientific replicate.
Capture one evaluation path and export verified records
Add declaration, execution and export commands for native CybORG checkpoint
evaluation. Retain resolved settings, checkpoint and source identities, completed
episodes, terminal status and explicit retry relationships. Export verifies the
retained evidence and maps it into the records contract from #35. The documentation
hub links the usage guide and explains how capture connects to import and collation.
Training and JAX checkpoint capture remain separate work
This slice evaluates native checkpoints serially. It does not capture training,
resume execution, evaluate JAX checkpoints or provide shared artifact retrieval.
Related: #31. Depends on #35.