feat(calendar): carry natural-language preferences through the loader - #45
Open
Luiz do Valle (luizvalle) wants to merge 3 commits into
Open
feat(calendar): carry natural-language preferences through the loader#45Luiz do Valle (luizvalle) wants to merge 3 commits into
Luiz do Valle (luizvalle) wants to merge 3 commits into
Conversation
First of a stack adding natural-language scheduling preferences alongside the existing numeric ones. This PR only teaches the schema and loader to carry the text; nothing reads it yet, so behaviour is unchanged. `CalendarAssistant` gains two optional fields: `preference_file`, a path relative to the task YAML, and `preference_md`, its contents. The loader inlines the file before `model_validate` because `CalendarTask` is frozen and the text cannot be assigned afterwards. Resolving it there also folds the preference into the task content hash, so editing a preference document correctly invalidates checkpointed runs of that task rather than silently reusing them. A declared but missing file raises `FileNotFoundError` naming both the reference and the path it resolved to, since a silently unpreferenced task would otherwise grade as if it had no preferences at all. Tasks that declare no `preference_file` are untouched: `preference_md` stays `None` and the numeric `preferences` list loads exactly as before. Tests cover resolution, relative paths, the missing-file error, hash sensitivity, and that all 21 shipped `small.yaml` tasks still load with numeric preferences intact. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4709c4ae-3295-4bbb-866b-dff62e7ad4d3
Luiz do Valle (luizvalle)
marked this pull request as draft
July 27, 2026 17:31
This was referenced Jul 27, 2026
The docstring explained that preference paths resolve relative to the declaring YAML file but never showed the shape, which is the part that is easy to get wrong. Adds a worked example and notes that yaml_path may be relative or absolute depending on how the benchmark was invoked. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4709c4ae-3295-4bbb-866b-dff62e7ad4d3
Luiz do Valle (luizvalle)
force-pushed
the
luizdovalle/soft-prefs-01-schema-loader
branch
from
July 27, 2026 20:33
378c589 to
1af7f26
Compare
13 tasks
13 tasks
The hash behind checkpoint dedup dumps the whole model with unset fields included, so adding preference_file and preference_md rewrote the hash of every existing task even though none of them set either one. All 21 small.yaml tasks changed identity, which strands any checkpoint or results.json written before this branch: the tasks look new and re-execute from scratch. Task gains an overridable _hash_exclude(), and CalendarTask drops both fields from the hash when no document is declared. The 21 tasks hash exactly as they did before, while a task that does declare a document keeps the text in its hash, so editing the prose still invalidates runs graded against the old wording. The old hashes are pinned in the test as literals rather than recomputed, since a test that derives them from the same code it is checking would follow the bug rather than catch it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4709c4ae-3295-4bbb-866b-dff62e7ad4d3
Luiz do Valle (luizvalle)
force-pushed
the
luizdovalle/soft-prefs-01-schema-loader
branch
from
July 28, 2026 00:16
8d3a584 to
de7668b
Compare
Luiz do Valle (luizvalle)
marked this pull request as ready for review
July 28, 2026 16:23
Collaborator
|
Looks good to me, thank you! |
Safoora Yousefi (safooray)
approved these changes
Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
Let a calendar task state its principal's preferences as prose in a Markdown file, instead of the numeric hourly table
preferencesholds today. This PR only carries the text; nothing reads it yet, so behavior is unchanged.Summary of changes
CalendarAssistantgainspreference_fileandpreference_md. The first is a path relative to the task YAML, the second its contents.The loader inlines the file before validation.
CalendarTaskis frozen, so the text has to be in the dict passed tomodel_validate. Doing it there also folds the document into the task content hash, so editing a preference invalidates cached runs graded against the old wording. A missing file raisesFileNotFoundErrornaming both the document and the task YAML.Tasks with no document hash exactly as they do on
main._hash_excludedrops both fields whenpreference_fileis unset, so existing checkpoints keep resolving.How to test
Tracked by #50.