diff --git a/.agents/skills/fine-tuning/SKILL.md b/.agents/skills/fine-tuning/SKILL.md index 73c91863..239465a4 100644 --- a/.agents/skills/fine-tuning/SKILL.md +++ b/.agents/skills/fine-tuning/SKILL.md @@ -59,6 +59,7 @@ Train language models with SFT, KTO, and GRPO locally or on supported cloud prov | Diagnose an offline SFT worker rejection | Read the exact retained `trainer.stderr.log`; closed phase/category/milestone labels locate a boundary, not a cause or retry authority; see `reference/sft-training.md` | | Native MLX prepared SFT candidate | Internal exact-token batch/loss and runtime comparison only; no native launch CLI or API provider yet; see `reference/native-local-training.md` | | Plan a derived training image | `python scripts/qualify_derived_training_image.py plan --config Trainers/image_profiles/.yaml` | +| Private GRPO constructor diagnosis | `python -B examples/training_qualification/grpo_constructor.py --help`; use a private config and separate reviewed build/diagnostic admissions; no public training authority; see `reference/grpo-training.md` and `docs/review/grpo-constructor-diagnostic.md` | | Warm Space scaffold | `python3 Trainers/cloud/scripts/manage_space.py render --template vllm_warm --output-dir /tmp/my-space --base-image ghcr.io//:` | | Warm Space deploy | `python3 Trainers/cloud/scripts/manage_space.py deploy --space-id / --template vllm_warm --base-image ghcr.io//: --hardware a10g-small --sleep-time 3600 --var BASE_MODEL=` | | ML training | `python tuner.py ml train --config Trainers/ml/configs/templates/regression.yaml` | diff --git a/.agents/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml b/.agents/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml index ca01872f..f0bc3a6a 100644 --- a/.agents/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml +++ b/.agents/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml @@ -8,7 +8,7 @@ model: runtime: provider: modal lock_path: tuner/execution/providers/modal/modal-runtime-v1.lock.json - expected_lock_sha256: 4f3fa81c9aeb9130258dec23f2b841e6dbdf2d686d1cf6b50249e743c3d496ca + expected_lock_sha256: 3bcd5d9ca2065a1462745e738e4e1d60f89e3c31829964bc2f054b795cf4b561 input: jsonl_path: /replace/with/private/prepared-dataset.jsonl mode: text diff --git a/.agents/skills/fine-tuning/reference/grpo-training.md b/.agents/skills/fine-tuning/reference/grpo-training.md index fde169e8..e1a600ca 100644 --- a/.agents/skills/fine-tuning/reference/grpo-training.md +++ b/.agents/skills/fine-tuning/reference/grpo-training.md @@ -2,6 +2,23 @@ Group Relative Policy Optimization — optimizes model behavior using reward functions against generated completions. +For the consumed 0d515 Modal trainer-setup failure and restricted, +constructor-only Modal diagnostic, see +`docs/review/grpo-constructor-diagnostic.md`. The pinned Unsloth Zoo 2026.1.2 +source requires an accelerator at import, so a CPU-only patched-constructor +probe cannot reach that boundary. The checked-in +`scripts/qualify_derived_training_image.py release plan-grpo-constructor-input` +command derives the exact four-row input. The private configured consumer is +`examples/training_qualification/grpo_constructor.py`; use its owner-only +example config and separate `prepare-model`, `plan`, `prepare`, `admit`, +`execute-once`, and `readback` review gates. It requires reviewed new +image/source and independent read-only retained snapshot custody. A diagnostic +claim is consumed; it grants no training replay or qualification. The old +deployment's superseded app generation is retained evidence, while the exact +old model Volume identity is rehydrated read-only. Generated Unsloth code is +not source-attested by pinned wheel files, so its type-site label remains +`OTHER` unless a future exact generated-code proof is added. + ## Public GRPO declarations and CPU preparation The provider-neutral `TrainingInputV1` accepts explicit method `grpo` with @@ -206,7 +223,7 @@ auto-select BF16 on BF16-capable hardware while the recipe requests FP16 can fail at `TRAINER_SETUP`. A closed `TypeError` marker alone cannot establish that this was the historical cause, and it does not authorize replay. -GRPO has a separate fixed 131-member offline inventory and 13-member outer +GRPO has a separate fixed 133-member offline inventory and 14-member outer closure. Existing SFT inventories remain unchanged. Maintenance uses the checked-in command, offline first because the outer closure binds its bytes: @@ -633,7 +650,7 @@ actual accepted wheel anchor is replaced by the actual newly built current-sourc wheel in bound build material, using the existing normal builder. Never invent a future SHA or embed a circular self-hash. Actual derived-image inventory and CPU GRPO qualification remain separate from the 396-distribution base observation. -The old SFT profiles and fixed SFT68/outer5 and GRPO131/outer13 inventories stay +The old SFT profiles and fixed SFT68/outer5 and GRPO133/outer14 inventories stay unchanged. Installed candidate tests require an actual fresh wheel, verify the runtime YAML/inventory are shipped and the source image profile is absent, and label their synthetic Git preplan observation as CPU control-flow evidence only. diff --git a/.claude/skills/fine-tuning/SKILL.md b/.claude/skills/fine-tuning/SKILL.md index 73c91863..239465a4 100644 --- a/.claude/skills/fine-tuning/SKILL.md +++ b/.claude/skills/fine-tuning/SKILL.md @@ -59,6 +59,7 @@ Train language models with SFT, KTO, and GRPO locally or on supported cloud prov | Diagnose an offline SFT worker rejection | Read the exact retained `trainer.stderr.log`; closed phase/category/milestone labels locate a boundary, not a cause or retry authority; see `reference/sft-training.md` | | Native MLX prepared SFT candidate | Internal exact-token batch/loss and runtime comparison only; no native launch CLI or API provider yet; see `reference/native-local-training.md` | | Plan a derived training image | `python scripts/qualify_derived_training_image.py plan --config Trainers/image_profiles/.yaml` | +| Private GRPO constructor diagnosis | `python -B examples/training_qualification/grpo_constructor.py --help`; use a private config and separate reviewed build/diagnostic admissions; no public training authority; see `reference/grpo-training.md` and `docs/review/grpo-constructor-diagnostic.md` | | Warm Space scaffold | `python3 Trainers/cloud/scripts/manage_space.py render --template vllm_warm --output-dir /tmp/my-space --base-image ghcr.io//:` | | Warm Space deploy | `python3 Trainers/cloud/scripts/manage_space.py deploy --space-id / --template vllm_warm --base-image ghcr.io//: --hardware a10g-small --sleep-time 3600 --var BASE_MODEL=` | | ML training | `python tuner.py ml train --config Trainers/ml/configs/templates/regression.yaml` | diff --git a/.claude/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml b/.claude/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml index ca01872f..f0bc3a6a 100644 --- a/.claude/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml +++ b/.claude/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml @@ -8,7 +8,7 @@ model: runtime: provider: modal lock_path: tuner/execution/providers/modal/modal-runtime-v1.lock.json - expected_lock_sha256: 4f3fa81c9aeb9130258dec23f2b841e6dbdf2d686d1cf6b50249e743c3d496ca + expected_lock_sha256: 3bcd5d9ca2065a1462745e738e4e1d60f89e3c31829964bc2f054b795cf4b561 input: jsonl_path: /replace/with/private/prepared-dataset.jsonl mode: text diff --git a/.claude/skills/fine-tuning/reference/grpo-training.md b/.claude/skills/fine-tuning/reference/grpo-training.md index fde169e8..e1a600ca 100644 --- a/.claude/skills/fine-tuning/reference/grpo-training.md +++ b/.claude/skills/fine-tuning/reference/grpo-training.md @@ -2,6 +2,23 @@ Group Relative Policy Optimization — optimizes model behavior using reward functions against generated completions. +For the consumed 0d515 Modal trainer-setup failure and restricted, +constructor-only Modal diagnostic, see +`docs/review/grpo-constructor-diagnostic.md`. The pinned Unsloth Zoo 2026.1.2 +source requires an accelerator at import, so a CPU-only patched-constructor +probe cannot reach that boundary. The checked-in +`scripts/qualify_derived_training_image.py release plan-grpo-constructor-input` +command derives the exact four-row input. The private configured consumer is +`examples/training_qualification/grpo_constructor.py`; use its owner-only +example config and separate `prepare-model`, `plan`, `prepare`, `admit`, +`execute-once`, and `readback` review gates. It requires reviewed new +image/source and independent read-only retained snapshot custody. A diagnostic +claim is consumed; it grants no training replay or qualification. The old +deployment's superseded app generation is retained evidence, while the exact +old model Volume identity is rehydrated read-only. Generated Unsloth code is +not source-attested by pinned wheel files, so its type-site label remains +`OTHER` unless a future exact generated-code proof is added. + ## Public GRPO declarations and CPU preparation The provider-neutral `TrainingInputV1` accepts explicit method `grpo` with @@ -206,7 +223,7 @@ auto-select BF16 on BF16-capable hardware while the recipe requests FP16 can fail at `TRAINER_SETUP`. A closed `TypeError` marker alone cannot establish that this was the historical cause, and it does not authorize replay. -GRPO has a separate fixed 131-member offline inventory and 13-member outer +GRPO has a separate fixed 133-member offline inventory and 14-member outer closure. Existing SFT inventories remain unchanged. Maintenance uses the checked-in command, offline first because the outer closure binds its bytes: @@ -633,7 +650,7 @@ actual accepted wheel anchor is replaced by the actual newly built current-sourc wheel in bound build material, using the existing normal builder. Never invent a future SHA or embed a circular self-hash. Actual derived-image inventory and CPU GRPO qualification remain separate from the 396-distribution base observation. -The old SFT profiles and fixed SFT68/outer5 and GRPO131/outer13 inventories stay +The old SFT profiles and fixed SFT68/outer5 and GRPO133/outer14 inventories stay unchanged. Installed candidate tests require an actual fresh wheel, verify the runtime YAML/inventory are shipped and the source image profile is absent, and label their synthetic Git preplan observation as CPU control-flow evidence only. diff --git a/.github/workflows/provider-free-conformance.yml b/.github/workflows/provider-free-conformance.yml index 1fab57a3..0fe4b0eb 100644 --- a/.github/workflows/provider-free-conformance.yml +++ b/.github/workflows/provider-free-conformance.yml @@ -1079,8 +1079,8 @@ jobs: import hashlib from tuner.runtime.offline_grpo_worker import verify_installed_grpo_members from tuner.runtime.packaged_grpo_worker_closure import load_packaged_grpo_worker_closure - assert len(verify_installed_grpo_members().closure.members) == 131 - assert len(load_packaged_grpo_worker_closure().members) == 13 + assert len(verify_installed_grpo_members().closure.members) == 133 + assert len(load_packaged_grpo_worker_closure().members) == 14 for package, relative in ( ("tuner.runtime", "manifests/offline-grpo-worker-v1.json"), ("tuner.runtime", "manifests/packaged-grpo-worker-v1.json"), diff --git a/.skills/fine-tuning/SKILL.md b/.skills/fine-tuning/SKILL.md index 73c91863..239465a4 100644 --- a/.skills/fine-tuning/SKILL.md +++ b/.skills/fine-tuning/SKILL.md @@ -59,6 +59,7 @@ Train language models with SFT, KTO, and GRPO locally or on supported cloud prov | Diagnose an offline SFT worker rejection | Read the exact retained `trainer.stderr.log`; closed phase/category/milestone labels locate a boundary, not a cause or retry authority; see `reference/sft-training.md` | | Native MLX prepared SFT candidate | Internal exact-token batch/loss and runtime comparison only; no native launch CLI or API provider yet; see `reference/native-local-training.md` | | Plan a derived training image | `python scripts/qualify_derived_training_image.py plan --config Trainers/image_profiles/.yaml` | +| Private GRPO constructor diagnosis | `python -B examples/training_qualification/grpo_constructor.py --help`; use a private config and separate reviewed build/diagnostic admissions; no public training authority; see `reference/grpo-training.md` and `docs/review/grpo-constructor-diagnostic.md` | | Warm Space scaffold | `python3 Trainers/cloud/scripts/manage_space.py render --template vllm_warm --output-dir /tmp/my-space --base-image ghcr.io//:` | | Warm Space deploy | `python3 Trainers/cloud/scripts/manage_space.py deploy --space-id / --template vllm_warm --base-image ghcr.io//: --hardware a10g-small --sleep-time 3600 --var BASE_MODEL=` | | ML training | `python tuner.py ml train --config Trainers/ml/configs/templates/regression.yaml` | diff --git a/.skills/fine-tuning/configs/qwen35_4b_token_profile.yaml b/.skills/fine-tuning/configs/qwen35_4b_token_profile.yaml index ca01872f..f0bc3a6a 100644 --- a/.skills/fine-tuning/configs/qwen35_4b_token_profile.yaml +++ b/.skills/fine-tuning/configs/qwen35_4b_token_profile.yaml @@ -8,7 +8,7 @@ model: runtime: provider: modal lock_path: tuner/execution/providers/modal/modal-runtime-v1.lock.json - expected_lock_sha256: 4f3fa81c9aeb9130258dec23f2b841e6dbdf2d686d1cf6b50249e743c3d496ca + expected_lock_sha256: 3bcd5d9ca2065a1462745e738e4e1d60f89e3c31829964bc2f054b795cf4b561 input: jsonl_path: /replace/with/private/prepared-dataset.jsonl mode: text diff --git a/.skills/fine-tuning/reference/grpo-training.md b/.skills/fine-tuning/reference/grpo-training.md index fde169e8..e1a600ca 100644 --- a/.skills/fine-tuning/reference/grpo-training.md +++ b/.skills/fine-tuning/reference/grpo-training.md @@ -2,6 +2,23 @@ Group Relative Policy Optimization — optimizes model behavior using reward functions against generated completions. +For the consumed 0d515 Modal trainer-setup failure and restricted, +constructor-only Modal diagnostic, see +`docs/review/grpo-constructor-diagnostic.md`. The pinned Unsloth Zoo 2026.1.2 +source requires an accelerator at import, so a CPU-only patched-constructor +probe cannot reach that boundary. The checked-in +`scripts/qualify_derived_training_image.py release plan-grpo-constructor-input` +command derives the exact four-row input. The private configured consumer is +`examples/training_qualification/grpo_constructor.py`; use its owner-only +example config and separate `prepare-model`, `plan`, `prepare`, `admit`, +`execute-once`, and `readback` review gates. It requires reviewed new +image/source and independent read-only retained snapshot custody. A diagnostic +claim is consumed; it grants no training replay or qualification. The old +deployment's superseded app generation is retained evidence, while the exact +old model Volume identity is rehydrated read-only. Generated Unsloth code is +not source-attested by pinned wheel files, so its type-site label remains +`OTHER` unless a future exact generated-code proof is added. + ## Public GRPO declarations and CPU preparation The provider-neutral `TrainingInputV1` accepts explicit method `grpo` with @@ -206,7 +223,7 @@ auto-select BF16 on BF16-capable hardware while the recipe requests FP16 can fail at `TRAINER_SETUP`. A closed `TypeError` marker alone cannot establish that this was the historical cause, and it does not authorize replay. -GRPO has a separate fixed 131-member offline inventory and 13-member outer +GRPO has a separate fixed 133-member offline inventory and 14-member outer closure. Existing SFT inventories remain unchanged. Maintenance uses the checked-in command, offline first because the outer closure binds its bytes: @@ -633,7 +650,7 @@ actual accepted wheel anchor is replaced by the actual newly built current-sourc wheel in bound build material, using the existing normal builder. Never invent a future SHA or embed a circular self-hash. Actual derived-image inventory and CPU GRPO qualification remain separate from the 396-distribution base observation. -The old SFT profiles and fixed SFT68/outer5 and GRPO131/outer13 inventories stay +The old SFT profiles and fixed SFT68/outer5 and GRPO133/outer14 inventories stay unchanged. Installed candidate tests require an actual fresh wheel, verify the runtime YAML/inventory are shipped and the source image profile is absent, and label their synthetic Git preplan observation as CPU control-flow evidence only. diff --git a/Trainers/grpo/runtime_v1.py b/Trainers/grpo/runtime_v1.py index 5370882d..6196663e 100644 --- a/Trainers/grpo/runtime_v1.py +++ b/Trainers/grpo/runtime_v1.py @@ -177,12 +177,21 @@ def build_grpo_config(config: dict, checkpoints_dir: Path, *, config_class, bf16 def train_grpo_core(*, model, tokenizer, reward_fn, formatted_dataset, - training_args, callbacks, trainer_class, resume=None, _runtime_milestone=None): + training_args, callbacks, trainer_class, resume=None, _runtime_milestone=None, + _constructor_only=False, _constructor_observer=None): + if type(_constructor_only) is not bool or ( + _constructor_only and (resume is not None or not callable(_constructor_observer)) + ) or (_constructor_observer is not None and not callable(_constructor_observer)): + raise ValueError("invalid constructor diagnostic selection") if _runtime_milestone is not None: _runtime_milestone("TRAINER_SETUP") - trainer = create_grpo_trainer(trainer_class=trainer_class, model=model, processing_class=tokenizer, - reward_funcs=reward_fn, args=training_args, - train_dataset=formatted_dataset, callbacks=callbacks) + kwargs = dict(model=model, processing_class=tokenizer, reward_funcs=reward_fn, + args=training_args, train_dataset=formatted_dataset, callbacks=callbacks) + if _constructor_observer is not None: + _constructor_observer(trainer_class, kwargs) + trainer = create_grpo_trainer(trainer_class=trainer_class, **kwargs) + if _constructor_only: + return trainer, None if _runtime_milestone is not None: _runtime_milestone("TRAIN_CALL") result = train_grpo_trainer(trainer, resume=resume) @@ -422,7 +431,8 @@ def unique(pairs): def run_grpo(*, config, dataset_bytes, reward_documents, reward_schema, model_snapshot, model_identity, output_root, workload, metric_sink=None, _dependency_factory=None, _packaged_artifacts=False, - qualification_evidence=None, _child_deadline=None, _runtime_milestone=None): + qualification_evidence=None, _child_deadline=None, _runtime_milestone=None, + _constructor_only=False, _constructor_observer=None): """Run the existing GRPO core with explicit immutable admitted inputs. This entry does not authenticate a public plan or attest model member @@ -431,6 +441,10 @@ def run_grpo(*, config, dataset_bytes, reward_documents, reward_schema, """ if _runtime_milestone is not None and not callable(_runtime_milestone): raise ValueError("invalid private runtime milestone callback") + if type(_constructor_only) is not bool or ( + _constructor_only and (not _packaged_artifacts or not callable(_constructor_observer)) + ) or (_constructor_observer is not None and not callable(_constructor_observer)): + raise ValueError("invalid constructor diagnostic selection") def mark(value): if _runtime_milestone is not None: _runtime_milestone(value) @@ -528,7 +542,10 @@ def on_log(self, args, state, control, logs=None, **kwargs): trainer, result = train_grpo_core(model=model, tokenizer=tokenizer, reward_fn=reward_fn, formatted_dataset=dataset, training_args=args, callbacks=callbacks, trainer_class=deps['trainer_class'], - **({"_runtime_milestone": mark} if _runtime_milestone is not None else {})) + **({"_runtime_milestone": mark} if _runtime_milestone is not None else {}), + _constructor_only=_constructor_only, _constructor_observer=_constructor_observer) + if _constructor_only: + return trainer mark("MEASUREMENT_RECORD") evidence = measurements.record(workload) if measurements is not None else None mark("SAVE") diff --git a/docs/review/grpo-constructor-diagnostic.md b/docs/review/grpo-constructor-diagnostic.md new file mode 100644 index 00000000..436a74ad --- /dev/null +++ b/docs/review/grpo-constructor-diagnostic.md @@ -0,0 +1,161 @@ +# Restricted GRPO constructor diagnostic + +## Status and evidence boundary + +The consumed run `modal-f5d1b7f10ce5d4196ccd13f9` and call +`fc-01M5CTF2FJ3AP4MCQ58646HCYG` remain untouched. Its closed +`GRPO_TRAINER_CHILD_EXEC_TYPE_TRAINER_SETUP` result locates a constructor +TypeError but does not identify its cause. The pinned published Unsloth Zoo +2026.1.2 source requires an accelerator at import; a CPU-only patched-class +probe cannot reach the constructor. No package, recipe, import-order or model +change is supported by that observation. + +This implementation is **not execution-ready**. It provides a restricted private +Modal diagnostic operator, a configured host consumer, and a distinct constructor-only result. It has no +public TrainingAPI method, training verb or CPU qualification receipt. There +has been no new provider call, image build/pull, model download, GPU attempt, +ML import, container or training operation in this iteration. + +## Checked-in path + +`python scripts/qualify_derived_training_image.py release +plan-grpo-constructor-input --plan-ready ... --recipe ... --dataset ... +--profiles-root Trainers/runtime_profiles` +derives a canonical four-row input from the normal GRPO compiler and original +JSONL. It checks the selected profile's actual bytes and model compatibility, +and binds the plan, recipe and JSONL hashes. The host rederives these bytes +before claiming a fresh attempt. The former Docker execution command and child +have been removed. + +`tuner.training.modal_grpo_constructor_diagnostic` is the private host adapter. +Its admission checks clean pushed host and engine commits with the exact +`synaptic-tuner` gitlink, exact wheel bytes against +the installed GRPO source closure, runtime release/deployment facts, pinned +scope, quote and independently fetched exact Hub member metadata. It binds a +new diagnostic image deployment **separately** from the retained snapshot +deployment. The old model-cache Volume is opened by its authenticated retained +name and ID and mounted read-only. A fresh claim precedes pricing, deployment +observation and Sandbox creation. The current new deployment layout is +rechecked. The host creates one named Sandbox from the exact new image ID with +one L40S, two CPUs, 8 GiB, a 120-second provider lifetime, blocked workload +network and no Secret. The host has a 240-second overall deadline from before +its first provider read, plus a bounded 30-second cleanup observation. It +streams a maximum 128 KiB dispatch and captures at most 8 KiB of closed result. +The supervisor redirects stdout and stderr to `/dev/null` before library +imports. A bounded ready/ack handshake keeps its result file accessible while +the Sandbox runs; the host makes one sized Modal FileIO read of at most 8193 +bytes. It never opens the SDK's line-buffered stdout or stderr. An absent or +late handshake leaves the claim consumed and the result inconclusive. + +The installed `packaged_grpo_constructor_child` supervisor verifies every +read-only retained member against the independent Git/LFS metadata and SHA256 +inventory, then uses the packaged private-copy helper. It starts the isolated +child through the pinned Python with `-I -B`, the same replacement environment +builder as normal packaged GRPO, the installed import guard, and the same +`run_grpo` path. The child's private selector returns immediately after the +six-argument trainer constructor. No `.train()`, generation, saving or optimizer +step is called by this path. Constructor internals still run under the bounded +Sandbox. Both processes suppress library output; only the closed result leaves +the Sandbox. + +The result includes fixed stage and category, six-argument signature binding, +patched-class predicate, requested precision versus actual model dtype, +model/tokenizer/reward/callback/dataset predicates, class/signature hashes, and +one conservative type-site label: `TRANSFORMERS_BASE` or `OTHER` for type +failures. The pinned Unsloth compiler writes trainer code at import time, so +the reviewed wheel-source hashes do not authenticate that generated region. +The generated patched-class shape is reported `UNKNOWN`. A positional MRO +match never labels an Unsloth region. The site mapper emits no exception text, +stack frame, line, path, argv or environment. Precision and object-shape +predicates can still separate hypotheses; `OTHER` remains inconclusive. + +The host independently reads the exact Sandbox ID's terminal state, then +terminates and rechecks its owned ID if needed. A late create result receives +bounded identity-checked cleanup. Create or cleanup uncertainty remains in a +consumed claim and the separate signed diagnostic receipt as distinct fields; +there is no automatic retry. The receipt explicitly has `cpu_qualified=false`, +`training_qualified=false`, `training_executed=false` and cannot be accepted by +the CPU reader or public training path. + +## Exact remaining admission + +Before the constructor GPU effect, root must review and bind: + +1. The final clean pushed engine and host superproject commits and gitlink; + the final refreshed complete GRPO offline and outer closure digests (fixed + 133 and 14 members), and Modal runtime lock SHA256; + the final wheel SHA256, runtime release digest, exact newly deployed Modal + image ID, pinned Python 3.11.14, SDK 1.5.4 and ML-stack inventory. The old + image does not contain this child, so a new image/release built by the normal + machinery is required. The base OCI pin remains + `sha256:5266c57be21059bfb407d80dc2f448868a5c2e2dbe7b2aa27780f48b48cbec39`. +2. The exact new deployment and the old retained snapshot deployment under + account/workspace `synaptic-labs`, environment `synaptic-smoke-v1`, each + app generation, image and three Volume IDs. Bind the old model-cache + Volume's name and ID separately; old calls, claims, read keys, Secrets and + resources are never reused as new execution authority or cleaned up. +3. An independent bounded pinned Hub metadata read for + `HuggingFaceTB/SmolLM2-135M-Instruct` revision + `a91318be21aeaf0879874faa161dcb40c68847e9`, complete expected member + names/sizes/Git or LFS hashes and reviewed SHA256s (at most 64 members, + 2 GiB), plus the current old Volume's verified read-only contents. A + manifest computed only from that Volume is insufficient. No missing member + may trigger a model download inside the diagnostic. +4. The exact four-row compiler input, recipe, plan and JSONL digests and + reviewed installed Unsloth/TRL/Transformers source SHA256s. The original + r3 input hash is historical. The retained `plan-ready.json` with the + checked-in candidate recipe and rows yielded historical v3 input SHA256 + `73b50f888df367fc0cffa18d7ea770b9690a79dfee80e36d97341d67aa0fea23` + (3336 bytes). This is a provider-free projection, not execution authority; + rederive against the exact final clean pushed source before admission. + The checked-in three-library source manifest must be bound by its final + SHA256 and each entry checked against its exact published wheel bytes. +5. One unused `grpo-diag-<24 hex>` attempt, distinct claim/catalog/receipt, + one L40S, two CPUs, 8 GiB, 120-second Sandbox lifetime, blocked workload + network, no Secret, 8 KiB closed result and a fresh scoped GPU quote under + an operator maximum of USD 1. The quote estimates GPU time and explicitly + excludes CPU, memory, storage and possible usage outside the timeout; it is + not a provider billing cap. + +The private consumer is `examples/training_qualification/grpo_constructor.py`. +Copy `examples/training_qualification/grpo-constructor.example.json` to an +owner-only private config, replace every placeholder with exact existing +paths/profile and two unused attempt refs, and use the locked CPython 3.11.14 +host with Modal SDK 1.5.4. The configured owner-only `hf_token_file` contains +only token bytes without a newline; the normal builder receives that named +value without searching ambient environment. The phases are separate: + +```text +python -B examples/training_qualification/grpo_constructor.py --config PRIVATE.json prepare-model +python -B examples/training_qualification/grpo_constructor.py --config PRIVATE.json plan +python -B examples/training_qualification/grpo_constructor.py --config PRIVATE.json prepare --reviewed-sha256 BUILD_SHA256 +python -B examples/training_qualification/grpo_constructor.py --config PRIVATE.json admit --reviewed-capture-sha256 CAPTURE_SHA256 --reviewed-model-sha256 MODEL_SHA256 +python -B examples/training_qualification/grpo_constructor.py --config PRIVATE.json execute-once --reviewed-sha256 EXECUTION_SHA256 --reviewed-capture-sha256 CAPTURE_SHA256 --reviewed-model-sha256 MODEL_SHA256 +python -B examples/training_qualification/grpo_constructor.py --config PRIVATE.json readback --reviewed-capture-sha256 CAPTURE_SHA256 --reviewed-model-sha256 MODEL_SHA256 +``` + +`prepare-model` uses bounded anonymous pinned Hub metadata and small Git-blob +reads. It never downloads LFS weights. Review its complete inventory and +SHA256 before `admit`. `plan` binds clean pushed source/gitlink, retained +release/deployment/Volume, four-row input, recipe, profile, library hashes and +build scope. `prepare` requires the reviewed plan digest before the normal +local build, deployment and unchanged CPU self-check. It retains the new wheel, +release, deployment, quote and CPU receipt for separate review. `admit` +rechecks the reviewed capture, upstream inventory and exact bindings before +writing a create-only GPU execution admission. `execute-once` requires that +reviewed digest, rederives the admission and consumes one claim before Modal +reads or Sandbox creation. `readback` verifies the distinct signed receipt +and prints only closed labels and hashes. The historical run supplies model +custody only. A new deployment may supersede the old app generation, so the +old facts are authenticated from retention and only its exact Volume is +rehydrated. The normal CPU self-check remains one CPU, 512 MiB, 120 seconds. + +## Provider-free verification + +The focused tests cover four-row compilation and mutations, independent model +metadata/source/wheel/scope checks, actual GRPO constructor-only selection, +no trainer train/generate/save path, conservative site classification, +actual interrupted child process-group termination, bounded result transfer, +identity-checked Sandbox cleanup and distinct signed receipt readback. These +CPU tests do not attest the pinned installed Unsloth class, the retained Modal +snapshot, a Modal GPU or a constructor cause. diff --git a/examples/training_qualification/grpo-constructor.example.json b/examples/training_qualification/grpo-constructor.example.json new file mode 100644 index 00000000..8471dab9 --- /dev/null +++ b/examples/training_qualification/grpo-constructor.example.json @@ -0,0 +1,23 @@ +{ + "schema_version": "synaptic-private-grpo-constructor-consumer/v1", + "engine_root": "/ABSOLUTE/CLEAN-PUSHED/synaptic-tuner", + "host_root": "/ABSOLUTE/CLEAN-PUSHED/HOST-WITH-EXACT-GITLINK", + "project_root": "/ABSOLUTE/GRPO-PROJECT", + "recipe_path": "/ABSOLUTE/GRPO-PROJECT/configuration/grpo.yaml", + "plan_ready_path": "/ABSOLUTE/GRPO-PROJECT/private/plan-ready.json", + "dataset_path": "/ABSOLUTE/GRPO-PROJECT/data/four-rows.jsonl", + "profiles_root": "/ABSOLUTE/CLEAN-PUSHED/synaptic-tuner/Trainers/runtime_profiles", + "private_root": "/ABSOLUTE/NEW-OWNER-PRIVATE/diagnostic-attempt", + "retained_root": "/ABSOLUTE/OLD-OWNER-PRIVATE/retained-run", + "retained_run": { + "run_id": "modal-f5d1b7f10ce5d4196ccd13f9", + "project_ref": "standalone-training" + }, + "modal_profile": "EXPLICIT-EXISTING-PROFILE", + "modal_environment": "synaptic-smoke-v1", + "hf_token_file": "/ABSOLUTE/OWNER-ONLY/HF-TOKEN-BYTES-WITHOUT-NEWLINE", + "model_ref": "HuggingFaceTB/SmolLM2-135M-Instruct", + "model_revision": "a91318be21aeaf0879874faa161dcb40c68847e9", + "cpu_attempt_ref": "cpu-qual-0123456789abcdef01234567", + "diagnostic_attempt_ref": "grpo-diag-0123456789abcdef01234567" +} diff --git a/examples/training_qualification/grpo_constructor.py b/examples/training_qualification/grpo_constructor.py new file mode 100644 index 00000000..7e24f21f --- /dev/null +++ b/examples/training_qualification/grpo_constructor.py @@ -0,0 +1,507 @@ +"""Private, review-gated GRPO constructor diagnostic consumer. + +Each effect phase consumes a separately reviewed canonical record. This is +not a TrainingAPI operation and never invokes the training Function. +""" +from __future__ import annotations + +import argparse +import hashlib +import json +import os +from pathlib import Path +import stat +import sys +from types import SimpleNamespace + +ENGINE = Path(__file__).resolve().parents[2] +if __package__ in {None, ""}: + sys.path.insert(0, str(ENGINE)) + +from synaptic_tuner.api.v1.results import TrainingRunRef +from tuner.cloud.grpo_constructor_diagnostic import derive_probe_input +from tuner.execution.foundation_v2.canonical import canonical_bytes +from tuner.execution.providers.modal.packaged_binding import ModalPackagedRuntimeFactsV1 +from tuner.execution.providers.modal.runtime_release_deployment import ModalRuntimeReleaseDeploymentFactsV1 +from tuner.project.context import ProjectContext +from tuner.runtime.packaged_grpo_constructor_child import pinned_library_sources, validate_model_inventory +from tuner.runtime.offline_grpo_worker import load_packaged_offline_grpo_worker_manifest +from tuner.runtime.packaged_grpo_worker_closure import load_packaged_grpo_worker_closure +from tuner.runtime.packaged_worker_closure import stable_read +from tuner.runtime.releases import parse_packaged_runtime_release +from tuner.training.contracts import ResourceSpec +from tuner.training.modal_grpo_constructor_diagnostic import ( + admit_modal_grpo_constructor, execute_modal_grpo_constructor_once, + prepare_pinned_model_manifest, read_modal_grpo_constructor_receipt, _source_pair, +) +from tuner.training.modal_host_runtime import ( + ModalHostRuntimeV1, ModalRuntimeQuoteV1, _CurrentPackagedDeploymentReader, +) +from tuner.training.modal_host_storage import ModalHostStorageV1 +from tuner.training.modal_local_preparation import private_directory, validate_local_launcher +from tuner.training.modal_recipe import ModalGRPORecipePlanV1, plan_modal_recipe +from tuner.training.modal_run_retention import load_retained_modal_run +from tuner.training.modal_retained_reader import _typed_inputs +from tuner.training.modal_standalone_runner import prepare_modal_standalone_local, _qualified_runtime + +_SCHEMA = "synaptic-private-grpo-constructor-consumer/v1" +_FIELDS = frozenset({"schema_version", "engine_root", "host_root", "project_root", "recipe_path", + "plan_ready_path", "dataset_path", "profiles_root", "private_root", "retained_root", + "retained_run", "modal_profile", "modal_environment", "hf_token_file", "model_ref", "model_revision", + "cpu_attempt_ref", "diagnostic_attempt_ref"}) +_HEX = frozenset("0123456789abcdef") +_MAX_CAPTURE = 2 * 1024 * 1024 + + +class ConsumerUnavailable(RuntimeError): + def __init__(self, phase): + self.phase = phase + super().__init__("grpo_constructor_consumer_unavailable") + + +class _Parser(argparse.ArgumentParser): + def error(self, message): + raise ConsumerUnavailable("ARGUMENTS") + + +def _hash(raw): + return hashlib.sha256(raw).hexdigest() + + +def _read(path, limit): + if not isinstance(path, Path) or not path.is_absolute(): + raise ValueError + info = path.lstat() + if (not stat.S_ISREG(info.st_mode) or info.st_nlink != 1 or info.st_uid != os.geteuid() + or stat.S_IMODE(info.st_mode) & 0o077): + raise ValueError + return stable_read(path, limit) + + +def _document(path, limit=_MAX_CAPTURE): + raw = _read(path, limit) + value = json.loads(raw) + if type(value) is not dict or canonical_bytes(value) != raw: + raise ValueError + return value, _hash(raw) + + +def _write(path, value): + raw = canonical_bytes(value) + if len(raw) > _MAX_CAPTURE: + raise ValueError + fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o600) + try: + with os.fdopen(fd, "wb", closefd=False) as stream: + stream.write(raw) + stream.flush() + os.fsync(stream.fileno()) + finally: + os.close(fd) + return _hash(raw) + + +def _config(path): + value, digest = _document(path, 16384) + if set(value) != _FIELDS or value["schema_version"] != _SCHEMA: + raise ValueError + for name in _FIELDS - {"schema_version", "retained_run", "modal_profile", "modal_environment", + "model_ref", "model_revision", "cpu_attempt_ref", "diagnostic_attempt_ref"}: + selected = value[name] + if type(selected) is not str or not Path(selected).is_absolute(): + raise ValueError + value[name] = Path(selected) + run = TrainingRunRef.from_dict(value["retained_run"]) + value["retained_run"] = run + for name in ("modal_profile", "modal_environment", "model_ref", "model_revision"): + if type(value[name]) is not str or not value[name]: + raise ValueError + if (type(value["cpu_attempt_ref"]) is not str + or not value["cpu_attempt_ref"].startswith("cpu-qual-") + or len(value["cpu_attempt_ref"]) != len("cpu-qual-") + 24 + or any(char not in _HEX for char in value["cpu_attempt_ref"][len("cpu-qual-"):]) + or type(value["diagnostic_attempt_ref"]) is not str + or not value["diagnostic_attempt_ref"].startswith("grpo-diag-") + or len(value["diagnostic_attempt_ref"]) != len("grpo-diag-") + 24 + or any(char not in _HEX for char in value["diagnostic_attempt_ref"][len("grpo-diag-"):])): + raise ValueError + return value, digest + + +def _paths(config): + return config["private_root"] + + +def _hf_token(config): + raw = _read(config["hf_token_file"], 4096) + if (not 1 <= len(raw) <= 512 or not raw.startswith(b"hf_") + or any(byte not in b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_" + for byte in raw)): + raise ValueError + return raw.decode("ascii") + + +def _plan(config, config_digest): + engine, host = _source_pair(config["engine_root"], config["host_root"]) + plan = plan_modal_recipe(config["recipe_path"], project_root=config["project_root"], + profiles_root=config["profiles_root"], source_root=config["engine_root"]) + if (type(plan) is not ModalGRPORecipePlanV1 + or plan.recipe.model.ref != config["model_ref"] + or plan.recipe.model.revision != config["model_revision"]): + raise ValueError + resource = ResourceSpec(plan.recipe.accelerator, plan.recipe.accelerator_count, + plan.recipe.timeout_seconds) + if (resource.accelerator != "L40S" or resource.accelerator_count != 1 + or resource.timeout_seconds != 120 or plan.recipe.maximum_cost_minor_units is None + or plan.recipe.maximum_cost_minor_units > 100): + raise ValueError + probe = derive_probe_input(recipe_path=config["recipe_path"], + plan_ready_path=config["plan_ready_path"], dataset_path=config["dataset_path"], + profiles_root=config["profiles_root"]) + retained = load_retained_modal_run(config["retained_root"], expected_run=config["retained_run"]) + old_inputs, old_release, old_deployment, old_facts, old_scope, *_ = _typed_inputs(retained) + old_model = old_facts.model_cache_volume_id + if (old_scope.environment_ref != config["modal_environment"] + or old_scope.account_ref != "synaptic-labs" + or old_scope.workspace_ref != "synaptic-labs" + or old_model not in dict(tuple(tuple(pair) for pair in old_inputs["volume_names"]))): + raise ValueError + candidate = {"schema_version": "synaptic-private-grpo-constructor-build-admission/v1", + "config_sha256": config_digest, "engine_commit": engine, "host_commit": host, + "offline_grpo_closure_digest": load_packaged_offline_grpo_worker_manifest().closure.closure_digest, + "outer_grpo_closure_digest": load_packaged_grpo_worker_closure().digest, + "modal_runtime_lock_sha256": _hash(stable_read(config["engine_root"] / + "tuner/execution/providers/modal/modal-runtime-v1.lock.json", 2 * 1024 * 1024)), + "python_version": "3.11.14", "modal_sdk_version": "1.5.4", + "recipe_plan_sha256": _hash(canonical_bytes(plan.to_dict())), + "probe_sha256": _hash(probe), "library_sources_sha256": _hash(canonical_bytes(pinned_library_sources())), + "profile_sha256": plan.profile_sha256, "runtime_intent_digest": plan.runtime_material_intent_digest, + "base_image": plan.profile_base_image, + "resource": {"accelerator": "L40S", "accelerator_count": 1, "timeout_seconds": 120}, + "maximum_cost_minor_units": plan.recipe.maximum_cost_minor_units, + "old_release_digest": old_release.manifest_digest, + "old_deployment_facts_digest": old_deployment.facts_digest, + "old_image_id": old_facts.image_id, "old_model_volume_id": old_model, + "old_model_volume_name": dict(tuple(tuple(pair) for pair in old_inputs["volume_names"]))[old_model], + "model_ref": config["model_ref"], "model_revision": config["model_revision"], + "cpu_attempt_ref": config["cpu_attempt_ref"], + "diagnostic_attempt_ref": config["diagnostic_attempt_ref"], + "account_ref": old_scope.account_ref, "workspace_ref": old_scope.workspace_ref, + "environment_ref": old_scope.environment_ref} + return candidate, plan, retained + + +def _checked(path, expected): + _, digest = _document(path) + if type(expected) is not str or len(expected) != 64 or digest != expected: + raise ValueError + return digest + + +def _quote(value): + if type(value) is not dict or set(value) != {"schema_version", "rate_key", "gpu_hourly_usd", + "gpu_only_timeout_estimate_minor_units", "maximum_cost_minor_units", "resource", + "excluded_billing_dimensions", "authorization_semantics", "quote_digest"}: + raise ValueError + if value["schema_version"] != "synaptic-modal-runtime-quote/v1": + raise ValueError + statement = {key: item for key, item in value.items() if key != "quote_digest"} + if _hash(canonical_bytes(statement)) != value["quote_digest"]: + raise ValueError + return ModalRuntimeQuoteV1(value["rate_key"], value["gpu_hourly_usd"], + value["gpu_only_timeout_estimate_minor_units"], value["maximum_cost_minor_units"], + ResourceSpec(**value["resource"]), value["quote_digest"], + tuple(value["excluded_billing_dimensions"]), value["authorization_semantics"]) + + +def _runtime(value, *, sdk, client, quote): + release, deployment, facts, volumes, key = _runtime_components(value) + binding = deployment.client_binding + reader = _CurrentPackagedDeploymentReader(sdk=sdk, client=client, + binding=binding, facts=facts, + names={item.spec.role: item.spec.name for item in deployment.volumes}, + secret_ids=tuple(sorted((item.spec.name, item.secret_id, item.spec.required_keys) + for item in deployment.secrets))) + return ModalHostRuntimeV1(release, deployment, facts, quote, volumes, key, reader) + + +def _runtime_components(value): + release = parse_packaged_runtime_release(value["release"]) + deployment = ModalRuntimeReleaseDeploymentFactsV1.from_dict(value["deployment_facts"]) + facts = ModalPackagedRuntimeFactsV1.from_dict(value["facts"]) + binding = deployment.client_binding + if (deployment.runtime_release_digest != release.manifest_digest + or ModalPackagedRuntimeFactsV1.from_release_deployment(deployment) != facts + or facts.client_binding != binding or facts.account_ref != "synaptic-labs" + or facts.workspace_ref != "synaptic-labs" or facts.environment_ref != "synaptic-smoke-v1"): + raise ValueError + volumes = tuple(tuple(pair) for pair in value["volume_names_by_id"]) + if dict(volumes) != {item.volume_id: item.spec.name for item in deployment.volumes}: + raise ValueError + key = bytes.fromhex(value["qualification_key_hex"]) + if len(key) != 32: + raise ValueError + return release, deployment, facts, volumes, key + + +def _readback(config, config_digest, root, reviewed, reviewed_capture, reviewed_model): + build, build_sha = _document(root / "build-admission.json") + if (build.get("schema_version") != "synaptic-private-grpo-constructor-build-admission/v1" + or build.get("config_sha256") != config_digest + or build.get("diagnostic_attempt_ref") != config["diagnostic_attempt_ref"] + or build.get("cpu_attempt_ref") != config["cpu_attempt_ref"] + or build.get("model_ref") != config["model_ref"] + or build.get("model_revision") != config["model_revision"] + or build.get("environment_ref") != config["modal_environment"]): + raise ValueError + _checked(root / "runtime-capture.json", reviewed_capture) + capture, capture_sha = _document(root / "runtime-capture.json") + if (capture.get("schema_version") != "synaptic-private-grpo-constructor-runtime-capture/v1" + or capture.get("build_admission_sha256") != build_sha + or capture.get("cpu_qualification", {}).get("training_executed") is not False): + raise ValueError + _quote(capture["quote"]) + release, deployment, facts, volumes, key = _runtime_components(capture["runtime"]) + if (release.package_digest != capture.get("wheel_sha256") + or build.get("account_ref") != facts.account_ref + or build.get("workspace_ref") != facts.workspace_ref + or build.get("environment_ref") != facts.environment_ref): + raise ValueError + _checked(root / "execution-admission.json", reviewed) + admission, _ = _document(root / "execution-admission.json") + model, model_sha = _document(root / "model-manifest.json") + validate_model_inventory(model) + if reviewed_model != model_sha: + raise ValueError + expected = {"schema_version": "synaptic-private-grpo-constructor-execution-admission/v1", + "capture_sha256": capture_sha, "model_manifest_sha256": model_sha, + "attempt_ref": config["diagnostic_attempt_ref"], + "source_commit": build["engine_commit"], "host_source_commit": build["host_commit"], + "wheel_sha256": capture["wheel_sha256"], + "runtime_release_digest": release.manifest_digest, + "deployment_facts_digest": deployment.facts_digest, "image_id": deployment.image_id, + "snapshot_deployment_facts_digest": build["old_deployment_facts_digest"], + "old_model_volume_id": build["old_model_volume_id"], + "old_model_volume_name": build["old_model_volume_name"], + "quote_digest": capture["quote"]["quote_digest"], + "account_ref": "synaptic-labs", "workspace_ref": "synaptic-labs", + "environment_ref": config["modal_environment"], + "gpu": "L40S", "gpu_count": 1, "cpu": 2, "memory_mib": 8192, + "provider_timeout_seconds": 120, "host_deadline_seconds": 240, + "cleanup_seconds": 30, "result_limit_bytes": 8192, "block_network": True, + "secrets": [], "training_executed": False} + if (set(admission) != set(expected) | {"dispatch_sha256"} + or any(admission.get(name) != value for name, value in expected.items()) + or type(admission.get("dispatch_sha256")) is not str + or len(admission["dispatch_sha256"]) != 64 + or any(char not in _HEX for char in admission["dispatch_sha256"])): + raise ValueError + runtime = SimpleNamespace(release=release, deployment_facts=deployment, + qualification_key=key) + with ModalHostStorageV1(root / "journal.sqlite3", "grpo-constructor-diagnostic", create=False) as storage: + receipt = read_modal_grpo_constructor_receipt(runtime, storage, config["diagnostic_attempt_ref"]) + body = receipt["body"] + bindings = {"dispatch_sha256": "dispatch_sha256", "source_commit": "source_commit", + "host_source_commit": "host_source_commit", "wheel_sha256": "wheel_sha256", + "runtime_release_digest": "runtime_release_digest", + "deployment_facts_digest": "deployment_facts_digest", "image_id": "image_id", + "snapshot_deployment_facts_digest": "snapshot_deployment_facts_digest", + "model_cache_volume_id": "old_model_volume_id", "quote_digest": "quote_digest"} + if any(body.get(receipt_key) != admission[record_key] + for receipt_key, record_key in bindings.items()): + raise ValueError + return _receipt_summary(receipt) + + +def _client(config): + import modal + if modal.__version__ != "1.5.4": + raise ValueError + from modal.config import config as modal_config + token_id = modal_config.get("token_id", profile=config["modal_profile"], use_env=False) + token_secret = modal_config.get("token_secret", profile=config["modal_profile"], use_env=False) + if any(type(value) is not str or not value.strip() for value in (token_id, token_secret)): + raise ValueError + return modal, modal.Client.from_credentials(token_id, token_secret) + + +def _read_capture(config, reviewed_capture, *, sdk, client): + path = _paths(config) / "runtime-capture.json" + _checked(path, reviewed_capture) + capture, _ = _document(path) + quote = _quote(capture["quote"]) + runtime = _runtime(capture["runtime"], sdk=sdk, client=client, quote=quote) + if capture["cpu_qualification"]["training_executed"] is not False: + raise ValueError + retained = load_retained_modal_run(config["retained_root"], expected_run=config["retained_run"]) + inputs, old_release, old_deployment, old_facts, old_scope, *_ = _typed_inputs(retained) + if (old_scope != runtime.deployment_facts.client_binding + or config["modal_environment"] != old_scope.environment_ref): + raise ValueError + snapshot = _runtime({"release": old_release.to_dict(), + "deployment_facts": old_deployment.to_dict(), "facts": old_facts.to_dict(), + "volume_names_by_id": inputs["volume_names"], + "qualification_key_hex": retained.qualification_key.hex()}, + sdk=sdk, client=client, quote=quote) + return capture, runtime, snapshot + + +def _admission(config, capture, runtime, snapshot, model): + return admit_modal_grpo_constructor(runtime=runtime, snapshot_runtime=snapshot, + engine_repository=config["engine_root"], host_repository=config["host_root"], + wheel_path=_paths(config) / "source-wheel.whl", recipe_path=config["recipe_path"], + plan_ready_path=config["plan_ready_path"], dataset_path=config["dataset_path"], + profiles_root=config["profiles_root"], + input_bytes=derive_probe_input(recipe_path=config["recipe_path"], + plan_ready_path=config["plan_ready_path"], dataset_path=config["dataset_path"], + profiles_root=config["profiles_root"]), model=model, + library_sources=pinned_library_sources(), attempt_ref=config["diagnostic_attempt_ref"]) + + +def _admission_record(config, admission, capture_sha, model_sha): + return {"schema_version": "synaptic-private-grpo-constructor-execution-admission/v1", + "capture_sha256": capture_sha, "model_manifest_sha256": model_sha, + "attempt_ref": admission.attempt_ref, "source_commit": admission.source_commit, + "host_source_commit": admission.host_source_commit, "wheel_sha256": admission.wheel_sha256, + "runtime_release_digest": admission.runtime_release_digest, + "deployment_facts_digest": admission.deployment_facts_digest, + "image_id": admission.image_id, + "snapshot_deployment_facts_digest": admission.snapshot_deployment_facts_digest, + "old_model_volume_id": admission.model_cache_volume_id, + "old_model_volume_name": admission.model_cache_volume_name, + "quote_digest": admission.quote_digest, "dispatch_sha256": _hash(admission.dispatch_bytes), + "account_ref": "synaptic-labs", "workspace_ref": "synaptic-labs", + "environment_ref": config["modal_environment"], + "gpu": "L40S", "gpu_count": 1, "cpu": 2, "memory_mib": 8192, + "provider_timeout_seconds": 120, "host_deadline_seconds": 240, + "cleanup_seconds": 30, "result_limit_bytes": 8192, "block_network": True, + "secrets": [], "training_executed": False} + + +def _receipt_summary(receipt): + body = receipt["body"] + return {"sha256": _hash(canonical_bytes(receipt)), "primary": body["primary"], + "cleanup": body["cleanup"], "failure_phase": body["failure_phase"], + "diagnostic": body["result"]} + + +def run(command, config_path, reviewed=None, reviewed_capture=None, reviewed_model=None): + config, config_digest = _config(config_path) + root = private_directory(_paths(config), create=command in {"plan", "prepare-model"}) + if command == "readback": + return _readback(config, config_digest, root, reviewed, reviewed_capture, reviewed_model) + if command == "prepare-model": + model = prepare_pinned_model_manifest(config["model_ref"], config["model_revision"]) + return _write(root / "model-manifest.json", model) + candidate, plan, _ = _plan(config, config_digest) + if command == "plan": + return _write(root / "build-admission.json", candidate) + build_record, build_digest = _document(root / "build-admission.json") + if build_record != candidate or (command == "prepare" and build_digest != reviewed): + raise ValueError + if command == "prepare": + if _document(root / "build-admission.json")[0] != candidate: + raise ValueError + validate_local_launcher() + context = ProjectContext.host(engine_root=config["engine_root"], project_root=config["project_root"]) + local = prepare_modal_standalone_local(plan=plan, context=context, state_root=root) + try: + wheel_bytes = stable_read(local.wheel, 256 * 1024 * 1024) + hf_token = _hf_token(config) + with ModalHostStorageV1(root / "journal.sqlite3", "grpo-constructor-diagnostic") as storage: + storage.attempts.claim("build-admission-" + config["cpu_attempt_ref"], + canonical_bytes(candidate)) + modal, _ = _client(config) + _, _, _, runtime, qualification = _qualified_runtime( + modal=modal, plan=plan, context=context, + resources=ResourceSpec("L40S", 1, 120), storage=storage, + effect_id=config["cpu_attempt_ref"], modal_profile=config["modal_profile"], + modal_environment=config["modal_environment"], + builder_cache_root=root / "modal-wheel-builder", local_inputs=local, + hf_token=hf_token) + if _hash(wheel_bytes) != runtime.release.package_digest: + raise ValueError + wheel_path = root / "source-wheel.whl" + fd = os.open(wheel_path, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o600) + try: + with os.fdopen(fd, "wb", closefd=False) as stream: + stream.write(wheel_bytes) + stream.flush() + os.fsync(stream.fileno()) + finally: + os.close(fd) + capture = {"schema_version": "synaptic-private-grpo-constructor-runtime-capture/v1", + "build_admission_sha256": reviewed, + "cpu_qualification": {"effect_id": config["cpu_attempt_ref"], + "output_sha256": qualification.output_sha256, + "provider_call_id": qualification.provider_call_id, + "training_executed": False, "gpu_qualified": False, + "receipt": qualification.receipt.to_dict()}, + "quote": runtime.quote.to_dict(), "runtime": { + "release": runtime.release.to_dict(), + "deployment_facts": runtime.deployment_facts.to_dict(), + "facts": runtime.facts.to_dict(), + "volume_names_by_id": [list(pair) for pair in runtime.volume_names_by_id], + "qualification_key_hex": runtime.qualification_key.hex()}, + "wheel_sha256": _hash(wheel_bytes)} + return _write(root / "runtime-capture.json", capture) + finally: + local.close() + _checked(root / "runtime-capture.json", reviewed_capture) + if command == "execute-once": + _checked(root / "execution-admission.json", reviewed) + model, model_sha = _document(root / "model-manifest.json") + validate_model_inventory(model) + if reviewed_model != model_sha: + raise ValueError + modal, client = _client(config) + capture, runtime, snapshot = _read_capture(config, reviewed_capture, sdk=modal, client=client) + if (capture["build_admission_sha256"] != _document(root / "build-admission.json")[1] + or capture["wheel_sha256"] != _hash(_read(root / "source-wheel.whl", 256 * 1024 * 1024))): + raise ValueError + admission = _admission(config, capture, runtime, snapshot, model) + record = _admission_record(config, admission, reviewed_capture, model_sha) + if command == "admit": + return _write(root / "execution-admission.json", record) + if command != "execute-once": + raise ValueError + if _document(root / "execution-admission.json")[0] != record: + raise ValueError + with ModalHostStorageV1(root / "journal.sqlite3", "grpo-constructor-diagnostic", create=False) as storage: + receipt = execute_modal_grpo_constructor_once(sdk=modal, client=client, + client_binding=runtime.deployment_facts.client_binding, + runtime=runtime, snapshot_runtime=snapshot, private_storage=storage, + admission=admission, engine_repository=config["engine_root"], + host_repository=config["host_root"], wheel_path=root / "source-wheel.whl", + recipe_path=config["recipe_path"], plan_ready_path=config["plan_ready_path"], + dataset_path=config["dataset_path"], profiles_root=config["profiles_root"]) + return _receipt_summary(receipt) + + +def main(argv=None): + parser = _Parser(description=__doc__) + parser.add_argument("--config", required=True, type=Path) + parser.add_argument("command", choices=("plan", "prepare-model", "prepare", "admit", "execute-once", "readback")) + parser.add_argument("--reviewed-sha256") + parser.add_argument("--reviewed-capture-sha256") + parser.add_argument("--reviewed-model-sha256") + try: + args = parser.parse_args(argv) + result = run(args.command, args.config, args.reviewed_sha256, + args.reviewed_capture_sha256, args.reviewed_model_sha256) + print(json.dumps({"status": "RECORDED", **(result if type(result) is dict else + {"sha256": result})}, sort_keys=True)) + return 0 + except SystemExit as error: + if error.code == 0: + return 0 + print(json.dumps({"status": "UNAVAILABLE", "phase": "ARGUMENTS"}, sort_keys=True)) + return 1 + except BaseException as error: + phase = error.phase if type(error) is ConsumerUnavailable else "UNAVAILABLE" + print(json.dumps({"status": "UNAVAILABLE", "phase": phase}, sort_keys=True)) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/pyproject.toml b/pyproject.toml index 7e366088..a311568d 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -75,7 +75,7 @@ namespaces = true "inference-dependencies.lock", ] "tuner.execution.providers.hf_jobs_provider_v1" = ["hf-parent-closure.json", "host-dependencies.in", "host-dependencies.lock", "model-dependencies.in", "model-dependencies.lock"] -"tuner.runtime" = ["manifests/offline-sft-worker-v1.json", "manifests/packaged-training-worker-v1.json", "manifests/offline-grpo-worker-v1.json", "manifests/packaged-grpo-worker-v1.json"] +"tuner.runtime" = ["manifests/offline-sft-worker-v1.json", "manifests/packaged-training-worker-v1.json", "manifests/offline-grpo-worker-v1.json", "manifests/packaged-grpo-worker-v1.json", "manifests/grpo-constructor-library-sources-v1.json"] "Trainers.sft.configs" = ["config.yaml"] # The Modal runtime lock and exact packaged SFT default config ship as assets. Broader diff --git a/scripts/qualify_derived_training_image.py b/scripts/qualify_derived_training_image.py index d519dea0..43b7be45 100644 --- a/scripts/qualify_derived_training_image.py +++ b/scripts/qualify_derived_training_image.py @@ -137,6 +137,10 @@ def _parser() -> argparse.ArgumentParser: local.add_argument("--docker-config", required=True, type=Path) local.add_argument("--execute", action="store_true") local_verify.add_argument("--evidence", required=True, type=Path) + grpo_input = release_actions.add_parser( + "plan-grpo-constructor-input", help="Derive the exact four-row constructor input from normal plan and recipe") + for flag in ("plan-ready", "recipe", "dataset", "profiles-root"): + grpo_input.add_argument("--" + flag, required=True, type=Path) return parser @@ -238,6 +242,14 @@ def main(argv: list[str] | None = None) -> int: _json({"schema_version": result["schema_version"], "status": result["status"]}) return 0 if args.command == "release": + if args.release_command == "plan-grpo-constructor-input": + from tuner.cloud.grpo_constructor_diagnostic import derive_probe_input + raw = derive_probe_input(recipe_path=args.recipe, plan_ready_path=args.plan_ready, + dataset_path=args.dataset, profiles_root=args.profiles_root) + _json({"schema_version": "synaptic-grpo-constructor-input-plan/v1", + "status": "PLAN_ONLY", "input_sha256": hashlib.sha256(raw).hexdigest(), + "input_size_bytes": len(raw), "row_count": 4}) + return 0 if args.release_command in {"qualify-local", "verify-local"}: inputs = dict(qualification_config_path=args.qualification_config, release_path=args.release, profile_path=args.config, build_receipt_path=args.build_receipt, candidate_path=args.candidate, diff --git a/scripts/regenerate_grpo_worker_closures.py b/scripts/regenerate_grpo_worker_closures.py index 523e2faa..3e6ee5ce 100644 --- a/scripts/regenerate_grpo_worker_closures.py +++ b/scripts/regenerate_grpo_worker_closures.py @@ -1,11 +1,14 @@ """Check fixed GRPO inventory hashes; initialization and refresh are explicit. Run offline first, then outer (the outer pins the offline manifest bytes). -The default never writes. Neither operation can discover or widen membership. +The default never writes. The sole reviewed additive migration admits the +constructor child and its three-library source commitment; no discovered +members are accepted. """ from __future__ import annotations import argparse +import hashlib import os from pathlib import Path import sys @@ -18,12 +21,44 @@ from tuner.runtime.offline_sft_worker import _canonical_json, _strict_document # noqa: E402 from tuner.runtime.offline_grpo_worker import ( # noqa: E402 captured_grpo_manifest_document, parse_offline_grpo_worker_manifest, + GRPO_OFFLINE_MEMBERS, GRPO_CLOSURE_SCHEMA, GRPO_CLOSURE_REF, + GRPO_ENTRYPOINT, GRPO_TRAINER_ENTRYPOINT, GRPO_BOOTSTRAP, GRPO_PACKAGED_SOURCE, ) +from tuner.runtime.offline_sft_worker import _parse_manifest, OfflineSFTWorkerError # noqa: E402 from tuner.runtime.packaged_grpo_worker_closure import ( # noqa: E402 captured_grpo_outer_document, parse_packaged_grpo_worker_closure, + GRPO_OUTER_MEMBERS, GRPO_OUTER_SCHEMA, ) from tuner.runtime.packaged_worker_closure import stable_read # noqa: E402 +_ADDED = frozenset({"tuner/runtime/packaged_grpo_constructor_child.py", + "tuner/runtime/manifests/grpo-constructor-library-sources-v1.json"}) + + +def _reviewed_previous(raw, *, kind): + """One-way policy migration for the single reviewed constructor child.""" + previous = _strict_document(raw) + if _canonical_json(previous) + b"\n" != raw: + raise ValueError + current = GRPO_OFFLINE_MEMBERS if kind == "offline" else GRPO_OUTER_MEMBERS + prior = tuple(path for path in current if path not in _ADDED) + if tuple(item.get("path") for item in previous["members"]) != prior: + raise ValueError + if kind == "offline": + _parse_manifest(previous, manifest_path=Path(GRPO_PACKAGED_SOURCE), + expected_digest=previous["closure_digest"], method="grpo", + _method_identity=(GRPO_CLOSURE_SCHEMA, GRPO_CLOSURE_REF, + GRPO_ENTRYPOINT, GRPO_TRAINER_ENTRYPOINT, GRPO_BOOTSTRAP), + _method_members=prior) + else: + if (set(previous) != {"schema_version", "members", "closure_digest"} + or previous["schema_version"] != GRPO_OUTER_SCHEMA + or hashlib.sha256(_canonical_json({"schema_version": GRPO_OUTER_SCHEMA, + "members": previous["members"]})).hexdigest() != previous["closure_digest"] + or any(set(item) != {"path", "size_bytes", "sha256"} for item in previous["members"])): + raise ValueError + return previous + def main(argv=None): parser = argparse.ArgumentParser(description=__doc__) @@ -53,10 +88,15 @@ def main(argv=None): return 0 previous_raw = stable_read(path, 1024 * 1024) previous = _strict_document(previous_raw) - if args.kind == "offline": - parse_offline_grpo_worker_manifest(previous_raw) - else: - parse_packaged_grpo_worker_closure(previous_raw, verify_sources=False) + migration = False + try: + if args.kind == "offline": + parse_offline_grpo_worker_manifest(previous_raw) + else: + parse_packaged_grpo_worker_closure(previous_raw, verify_sources=False) + except (ValueError, OfflineSFTWorkerError): + previous = _reviewed_previous(previous_raw, kind=args.kind) + migration = True if previous_raw == payload: print("CURRENT") return 0 @@ -65,7 +105,14 @@ def main(argv=None): return 3 # Only content hashes and byte totals change; capture uses the same fixed # reviewed identities and path order. Refuse any metadata substitution. - for old, new in zip(previous["members"], updated["members"], strict=True): + refreshed = {item["path"]: item for item in updated["members"]} + additions = set(refreshed) - {item["path"] for item in previous["members"]} + expected_additions = _ADDED if args.kind == "offline" else _ADDED - { + "tuner/runtime/manifests/grpo-constructor-library-sources-v1.json"} + if migration and additions != expected_additions: + raise ValueError + for old in previous["members"]: + new = refreshed[old["path"]] if {key: value for key, value in old.items() if key not in {"size_bytes", "sha256"}} != { key: value for key, value in new.items() if key not in {"size_bytes", "sha256"}}: raise ValueError diff --git a/tests/cloud/test_grpo_constructor_consumer.py b/tests/cloud/test_grpo_constructor_consumer.py new file mode 100644 index 00000000..abea2466 --- /dev/null +++ b/tests/cloud/test_grpo_constructor_consumer.py @@ -0,0 +1,253 @@ +"""Provider-free checks for the private constructor consumer's review gates.""" +from __future__ import annotations + +import hashlib +from types import SimpleNamespace + +import pytest + +from examples.training_qualification import grpo_constructor as consumer +from tuner.execution.foundation_v2.canonical import canonical_bytes + + +def _configuration(tmp_path): + fields = {"schema_version": consumer._SCHEMA, + "engine_root": str(tmp_path / "engine"), "host_root": str(tmp_path / "host"), + "project_root": str(tmp_path / "project"), "recipe_path": str(tmp_path / "recipe.yaml"), + "plan_ready_path": str(tmp_path / "plan.json"), "dataset_path": str(tmp_path / "rows.jsonl"), + "profiles_root": str(tmp_path / "profiles"), "private_root": str(tmp_path / "private"), + "retained_root": str(tmp_path / "retained"), + "retained_run": {"run_id": "modal-historical", "project_ref": "standalone-training"}, + "modal_profile": "reviewed-profile", "modal_environment": "synaptic-smoke-v1", + "hf_token_file": str(tmp_path / "hf-token"), + "model_ref": "HuggingFaceTB/SmolLM2-135M-Instruct", + "model_revision": "a91318be21aeaf0879874faa161dcb40c68847e9", + "cpu_attempt_ref": "cpu-qual-" + "a" * 24, + "diagnostic_attempt_ref": "grpo-diag-" + "b" * 24} + path = tmp_path / "config.json" + consumer._write(path, fields) + return path + + +def test_private_config_and_canonical_records_reject_mutations(tmp_path): + config = _configuration(tmp_path) + parsed, digest = consumer._config(config) + assert parsed["engine_root"] == tmp_path / "engine" + assert len(digest) == 64 + record = tmp_path / "record.json" + selected = consumer._write(record, {"schema_version": "candidate", "value": 1}) + assert consumer._document(record)[1] == selected + with pytest.raises(FileExistsError): + consumer._write(record, {"schema_version": "candidate", "value": 2}) + record.chmod(0o644) + with pytest.raises(ValueError): + consumer._document(record) + + +def test_explicit_token_file_is_private_and_exact(tmp_path, monkeypatch): + config = _configuration(tmp_path) + parsed, _ = consumer._config(config) + token = parsed["hf_token_file"] + monkeypatch.setenv("HF_TOKEN", "hf_ambient_must_not_be_used") + consumer._write(token, {"wrong": "format"}) + with pytest.raises(ValueError): + consumer._hf_token(parsed) + token.unlink() + token.write_bytes(b"hf_exactbytes") + token.chmod(0o600) + assert consumer._hf_token(parsed) == "hf_exactbytes" + token.write_bytes(b"hf_exactbytes\n") + with pytest.raises(ValueError): + consumer._hf_token(parsed) + + +def test_build_review_digest_precedes_launcher_or_provider(monkeypatch, tmp_path): + config = _configuration(tmp_path) + candidate = {"schema_version": "build", "source_commit": "a" * 40} + monkeypatch.setattr(consumer, "_plan", lambda *args: (candidate, object(), object())) + monkeypatch.setattr(consumer, "validate_local_launcher", lambda: pytest.fail("launcher reached")) + monkeypatch.setattr(consumer, "_client", lambda config: pytest.fail("client reached")) + first = consumer.run("plan", config) + assert first == hashlib.sha256(canonical_bytes(candidate)).hexdigest() + with pytest.raises(ValueError): + consumer.run("prepare", config, reviewed="0" * 64) + assert consumer._document(tmp_path / "private" / "build-admission.json")[1] == first + + +def test_execution_review_digest_precedes_metadata_or_client(monkeypatch, tmp_path): + config = _configuration(tmp_path) + candidate = {"schema_version": "build", "source_commit": "a" * 40} + monkeypatch.setattr(consumer, "_plan", lambda *args: (candidate, object(), object())) + consumer.run("plan", config) + root = tmp_path / "private" + capture = consumer._write(root / "runtime-capture.json", {"schema_version": "capture"}) + model = consumer._write(root / "model-manifest.json", {"schema_version": "model"}) + consumer._write(root / "execution-admission.json", {"schema_version": "execution"}) + monkeypatch.setattr(consumer, "validate_model_inventory", lambda value: None) + monkeypatch.setattr(consumer, "_client", lambda config: pytest.fail("client reached")) + monkeypatch.setattr(consumer, "_admission", lambda *args: pytest.fail("metadata reached")) + with pytest.raises(ValueError): + consumer.run("execute-once", config, reviewed="0" * 64, + reviewed_capture=capture, reviewed_model=model) + + +def test_help_is_a_success_without_opening_config(capsys): + assert consumer.main(["--help"]) == 0 + assert "UNAVAILABLE" not in capsys.readouterr().out + + +def test_prepare_composes_normal_cpu_qualification_with_explicit_token(monkeypatch, tmp_path): + config = _configuration(tmp_path) + token = tmp_path / "hf-token" + token.write_bytes(b"hf_explicit_fixture") + token.chmod(0o600) + candidate = {"schema_version": "build", "source_commit": "a" * 40} + plan = object() + monkeypatch.setattr(consumer, "_plan", lambda *args: (candidate, plan, object())) + reviewed = consumer.run("plan", config) + wheel = tmp_path / "wheel.whl" + wheel.write_bytes(b"fixed-wheel") + wheel_sha = hashlib.sha256(b"fixed-wheel").hexdigest() + closed = [] + monkeypatch.setattr(consumer, "validate_local_launcher", lambda: None) + monkeypatch.setattr(consumer, "prepare_modal_standalone_local", lambda **kwargs: + SimpleNamespace(wheel=wheel, close=lambda: closed.append(True))) + monkeypatch.setattr(consumer, "_client", lambda config: (SimpleNamespace(__version__="1.5.4"), object())) + def qualify(**kwargs): + assert kwargs["plan"] is plan + assert kwargs["hf_token"] == "hf_explicit_fixture" + assert kwargs["effect_id"] == "cpu-qual-" + "a" * 24 + assert kwargs["resources"].accelerator == "L40S" + release = SimpleNamespace(package_digest=wheel_sha, to_dict=lambda: {"release": "fixed"}) + runtime = SimpleNamespace(release=release, + deployment_facts=SimpleNamespace(to_dict=lambda: {"deployment": "fixed"}), + facts=SimpleNamespace(to_dict=lambda: {"facts": "fixed"}), + quote=SimpleNamespace(to_dict=lambda: {"quote": "fixed"}), + volume_names_by_id=(("vo-new", "new-volume"),), qualification_key=b"q" * 32) + qualification = SimpleNamespace(output_sha256="1" * 64, + provider_call_id="fc-fixture", receipt=SimpleNamespace(to_dict=lambda: {"receipt": "fixed"})) + return None, None, None, runtime, qualification + monkeypatch.setattr(consumer, "_qualified_runtime", qualify) + captured_sha = consumer.run("prepare", config, reviewed=reviewed) + capture, digest = consumer._document(tmp_path / "private" / "runtime-capture.json") + assert digest == captured_sha + assert capture["wheel_sha256"] == wheel_sha + assert capture["cpu_qualification"]["training_executed"] is False + assert closed == [True] + + +def test_admit_then_execute_rederives_exact_reviewed_record(monkeypatch, tmp_path): + from tuner.training.modal_host_storage import ModalHostStorageV1 + config = _configuration(tmp_path) + candidate = {"schema_version": "build", "source_commit": "a" * 40} + monkeypatch.setattr(consumer, "_plan", lambda *args: (candidate, object(), object())) + build_sha = consumer.run("plan", config) + root = tmp_path / "private" + wheel = root / "source-wheel.whl" + wheel.write_bytes(b"fixed-wheel") + wheel.chmod(0o600) + wheel_sha = hashlib.sha256(b"fixed-wheel").hexdigest() + capture = {"build_admission_sha256": build_sha, "wheel_sha256": wheel_sha} + capture_sha = consumer._write(root / "runtime-capture.json", capture) + model_sha = consumer._write(root / "model-manifest.json", {"members": []}) + monkeypatch.setattr(consumer, "validate_model_inventory", lambda model: None) + sdk, client = object(), object() + runtime = SimpleNamespace(deployment_facts=SimpleNamespace(client_binding=object())) + snapshot = object() + monkeypatch.setattr(consumer, "_client", lambda config: (sdk, client)) + monkeypatch.setattr(consumer, "_read_capture", lambda *args, **kwargs: + (capture, runtime, snapshot)) + admission = SimpleNamespace(attempt_ref="grpo-diag-" + "b" * 24, + source_commit="a" * 40, host_source_commit="c" * 40, + wheel_sha256=wheel_sha, runtime_release_digest="d" * 64, + deployment_facts_digest="e" * 64, image_id="im-fixed", + snapshot_deployment_facts_digest="f" * 64, + model_cache_volume_id="vo-old", model_cache_volume_name="old-cache", + quote_digest="1" * 64, dispatch_bytes=b"exact-dispatch") + calls = [] + def admit(*args, **kwargs): + calls.append("admit") + return admission + monkeypatch.setattr(consumer, "_admission", admit) + reviewed = consumer.run("admit", config, reviewed_capture=capture_sha, + reviewed_model=model_sha) + with ModalHostStorageV1(root / "journal.sqlite3", "grpo-constructor-diagnostic"): + pass + def execute(**kwargs): + assert kwargs["sdk"] is sdk and kwargs["client"] is client + assert kwargs["admission"] is admission + assert kwargs["snapshot_runtime"] is snapshot + calls.append("execute") + return {"body": {"primary": "INDETERMINATE", "cleanup": "TERMINAL_CONFIRMED", + "failure_phase": "OUTPUT", "result": None}, "tag_hex": "00"} + monkeypatch.setattr(consumer, "execute_modal_grpo_constructor_once", execute) + outcome = consumer.run("execute-once", config, reviewed=reviewed, + reviewed_capture=capture_sha, reviewed_model=model_sha) + assert outcome["primary"] == "INDETERMINATE" + assert calls == ["admit", "admit", "execute"] + + +def test_readback_restart_uses_only_reviewed_retained_material(monkeypatch, tmp_path): + from tuner.training.modal_grpo_constructor_diagnostic import _catalog, _receipt + from tuner.training.modal_host_storage import ModalHostStorageV1 + + config_path = _configuration(tmp_path) + config, config_sha = consumer._config(config_path) + root = consumer.private_directory(config["private_root"], create=True) + build = {"schema_version": "synaptic-private-grpo-constructor-build-admission/v1", + "config_sha256": config_sha, "diagnostic_attempt_ref": config["diagnostic_attempt_ref"], + "cpu_attempt_ref": config["cpu_attempt_ref"], "model_ref": config["model_ref"], + "model_revision": config["model_revision"], "environment_ref": config["modal_environment"], + "account_ref": "synaptic-labs", "workspace_ref": "synaptic-labs", + "engine_commit": "a" * 40, "host_commit": "b" * 40, + "old_deployment_facts_digest": "c" * 64, + "old_model_volume_id": "vo-old", "old_model_volume_name": "old-cache"} + build_sha = consumer._write(root / "build-admission.json", build) + model_sha = consumer._write(root / "model-manifest.json", {"members": []}) + wheel_sha = "d" * 64 + capture = {"schema_version": "synaptic-private-grpo-constructor-runtime-capture/v1", + "build_admission_sha256": build_sha, "wheel_sha256": wheel_sha, + "cpu_qualification": {"training_executed": False}, "quote": {"quote_digest": "e" * 64}, + "runtime": {"retained": True}} + capture_sha = consumer._write(root / "runtime-capture.json", capture) + key = b"k" * 32 + release = SimpleNamespace(manifest_digest="f" * 64, package_digest=wheel_sha) + deployment = SimpleNamespace(facts_digest="1" * 64, image_id="im-new") + facts = SimpleNamespace(account_ref="synaptic-labs", workspace_ref="synaptic-labs", + environment_ref="synaptic-smoke-v1") + monkeypatch.setattr(consumer, "_quote", lambda value: object()) + monkeypatch.setattr(consumer, "_runtime_components", lambda value: + (release, deployment, facts, (), key)) + monkeypatch.setattr(consumer, "validate_model_inventory", lambda value: None) + admission = SimpleNamespace(attempt_ref=config["diagnostic_attempt_ref"], + dispatch_bytes=b"exact-dispatch", source_commit=build["engine_commit"], + host_source_commit=build["host_commit"], wheel_sha256=wheel_sha, + runtime_release_digest=release.manifest_digest, + deployment_facts_digest=deployment.facts_digest, image_id=deployment.image_id, + snapshot_deployment_facts_digest=build["old_deployment_facts_digest"], + model_cache_volume_id="vo-old", model_cache_volume_name="old-cache", + quote_digest=capture["quote"]["quote_digest"]) + record = consumer._admission_record(config, admission, capture_sha, model_sha) + reviewed = consumer._write(root / "execution-admission.json", record) + runtime = SimpleNamespace(release=release, deployment_facts=deployment, + qualification_key=key) + receipt = _receipt(runtime, admission, sandbox_id="sb-fixed", primary="REJECTED", + cleanup="TERMINAL_CONFIRMED", result=None, terminal=1, + failure_phase="OUTPUT") + with ModalHostStorageV1(root / "journal.sqlite3", "grpo-constructor-diagnostic") as storage: + _catalog(storage, "modal-grpo-constructor-results").publish_if_absent( + config["diagnostic_attempt_ref"], receipt) + (tmp_path / "recipe.yaml").write_text("changed recipe") + (tmp_path / "engine").mkdir() + (tmp_path / "engine" / "dirty").write_text("advanced source") + monkeypatch.delenv("HF_TOKEN", raising=False) + for forbidden in ("_plan", "_client", "_admission", "execute_modal_grpo_constructor_once"): + monkeypatch.setattr(consumer, forbidden, + lambda *args, **kwargs: pytest.fail("readback reached live authority")) + result = consumer.run("readback", config_path, reviewed=reviewed, + reviewed_capture=capture_sha, reviewed_model=model_sha) + assert result["primary"] == "REJECTED" + assert result["cleanup"] == "TERMINAL_CONFIRMED" + with pytest.raises(ValueError): + consumer.run("readback", config_path, reviewed="0" * 64, + reviewed_capture=capture_sha, reviewed_model=model_sha) diff --git a/tests/cloud/test_grpo_constructor_diagnostic.py b/tests/cloud/test_grpo_constructor_diagnostic.py new file mode 100644 index 00000000..1ca24467 --- /dev/null +++ b/tests/cloud/test_grpo_constructor_diagnostic.py @@ -0,0 +1,736 @@ +from __future__ import annotations + +import hashlib +import json +import os +from pathlib import Path +import sys +import time +from types import SimpleNamespace +from urllib.parse import urlparse + +import pytest + +from tuner.cloud.derived_training_image import DerivedTrainingImageError +from tuner.cloud.grpo_constructor_diagnostic import derive_probe_input, verify_probe_input +from tuner.runtime import packaged_grpo_constructor_child as child +from tuner.training import modal_grpo_constructor_diagnostic as host + + +def _planned_input(tmp_path: Path): + import yaml + from synaptic_tuner.api.v1.training_input import TrainingInputV1 + from synaptic_tuner.api.v1.training_sources import PreparedTrainingInputIdentity + from tuner.training.grpo_compilation import compile_grpo_input, grpo_prepared_semantic_digest + root = Path(__file__).resolve().parents[2] + recipe = root / "examples/grpo_captured_candidate/recipe.yaml" + dataset = root / "examples/grpo_captured_candidate/rows.jsonl" + data = yaml.safe_load(recipe.read_bytes()) + raw = dataset.read_bytes() + digest = hashlib.sha256(raw).hexdigest() + semantic = grpo_prepared_semantic_digest(content_digest=digest, size_bytes=len(raw), + row_count=4, prompt_column="prompt") + prepared = PreparedTrainingInputIdentity(ref="prepared://sha256/" + semantic, + revision=semantic, content_digest=digest, size_bytes=len(raw), format="synaptic-grpo-prompts/v1") + training = TrainingInputV1.from_dict(data["training_input"]) + compiled = compile_grpo_input(training, prepared) + from tuner.runtime_profiles import load_runtime_profile + profile = load_runtime_profile(data["job"]["runtime_profile"], root / "Trainers/runtime_profiles") + plan = tmp_path / "plan.json" + plan.write_text(json.dumps({"data": {"training_input": training.to_dict(), + "prepared_dataset": prepared.to_dict(), "core_workload_digest": compiled.fingerprint, + "runtime_profile": data["job"]["runtime_profile"], + "profile_sha256": profile.profile_sha256, + "resource_request": {"accelerator": data["job"]["accelerator"], + "accelerator_count": data["job"]["accelerator_count"], + "timeout_seconds": data["job"]["timeout_seconds"]}, + "limits": data["limits"]}})) + return recipe, dataset, plan, compiled + + +def test_exact_four_row_input_and_dataset_mutation(tmp_path): + recipe, dataset, plan, compiled = _planned_input(tmp_path) + raw = derive_probe_input(recipe_path=recipe, plan_ready_path=plan, dataset_path=dataset) + value = verify_probe_input(raw) + assert value["row_count"] == 4 + assert value["workload"] == compiled.document + assert len(raw) <= 65536 + changed = tmp_path / "changed.jsonl" + changed.write_bytes(dataset.read_bytes().replace(b"blue", b"green", 1)) + with pytest.raises(DerivedTrainingImageError): + derive_probe_input(recipe_path=recipe, plan_ready_path=plan, dataset_path=changed) + with pytest.raises(DerivedTrainingImageError): + verify_probe_input(raw.replace(b"\n", b" \n", 1)) + + +def test_plan_profile_resource_and_recipe_are_bound_before_dispatch(tmp_path): + recipe, dataset, plan, _ = _planned_input(tmp_path) + profiles_root = Path(__file__).resolve().parents[2] / "Trainers/runtime_profiles" + original = derive_probe_input(recipe_path=recipe, plan_ready_path=plan, + dataset_path=dataset, profiles_root=profiles_root) + document = json.loads(plan.read_bytes()) + for field, value in (("profile_sha256", "sha256:" + "0" * 64), + ("resource_request", {"accelerator": "A100", "accelerator_count": 1, + "timeout_seconds": 900})): + mutated = json.loads(plan.read_bytes()) + mutated["data"][field] = value + plan.write_text(json.dumps(mutated)) + with pytest.raises(DerivedTrainingImageError): + derive_probe_input(recipe_path=recipe, plan_ready_path=plan, + dataset_path=dataset, profiles_root=profiles_root) + plan.write_text(json.dumps(document)) + assert derive_probe_input(recipe_path=recipe, plan_ready_path=plan, + dataset_path=dataset, profiles_root=profiles_root) == original + changed_recipe = tmp_path / "changed-recipe.yaml" + changed_recipe.write_bytes(recipe.read_bytes().replace(b"fp16: true", b"fp16: false", 1)) + with pytest.raises(DerivedTrainingImageError): + derive_probe_input(recipe_path=changed_recipe, plan_ready_path=plan, + dataset_path=dataset, profiles_root=profiles_root) + + +def test_model_inventory_rejects_metadata_and_missing_member_mutations(): + data = b"model" + model = {"ref": "Example/Model", "revision": "a" * 40, + "tokenizer_revision": "a" * 40, "members": [{"path": "config.json", + "size_bytes": len(data), "kind": "git", + "upstream_digest": hashlib.sha1(b"blob 5\0" + data).hexdigest(), + "sha256": hashlib.sha256(data).hexdigest()}]} + assert child.validate_model_inventory(model) == (("config.json", 5, hashlib.sha256(data).hexdigest()),) + for key, value in (("path", "../config.json"), ("size_bytes", 6), + ("upstream_digest", "0" * 40), ("sha256", "0" * 64)): + changed = {**model, "members": [{**model["members"][0], key: value}]} + if key == "upstream_digest": + assert changed["members"][0]["upstream_digest"] != model["members"][0]["upstream_digest"] + elif key in {"size_bytes", "sha256"}: + assert child.validate_model_inventory(changed) != child.validate_model_inventory(model) + else: + with pytest.raises(ValueError): + child.validate_model_inventory(changed) + with pytest.raises(ValueError): + child.validate_model_inventory({**model, "members": model["members"] * 2}) + + +def test_host_and_engine_source_require_clean_pushed_exact_gitlink(monkeypatch, tmp_path): + engine = tmp_path / "engine" + parent = tmp_path / "host" + engine.mkdir() + parent.mkdir() + commits = {engine: "a" * 40, parent: "b" * 40} + monkeypatch.setattr(host, "inspect_git_source", lambda repo: SimpleNamespace( + dirty=False, pushed=True, commit=commits[repo])) + monkeypatch.setattr(host, "_git", lambda *args: "160000 commit " + "a" * 40 + "\tsynaptic-tuner") + assert host._source_pair(engine, parent) == ("a" * 40, "b" * 40) + commits[parent] = "c" * 40 + monkeypatch.setattr(host, "_git", lambda *args: "160000 commit " + "d" * 40 + "\tsynaptic-tuner") + with pytest.raises(ValueError): + host._source_pair(engine, parent) + monkeypatch.setattr(host, "inspect_git_source", lambda repo: SimpleNamespace( + dirty=repo == engine, pushed=True, commit=commits[repo])) + with pytest.raises(ValueError): + host._source_pair(engine, parent) + + +def test_dispatch_binds_input_source_image_and_model(monkeypatch, tmp_path): + recipe, dataset, plan, _ = _planned_input(tmp_path) + raw = derive_probe_input(recipe_path=recipe, plan_ready_path=plan, dataset_path=dataset) + monkeypatch.setattr(child, "parse_packaged_runtime_release", lambda value: value) + model = {"ref": "Example/Model", "revision": "a" * 40, + "tokenizer_revision": "a" * 40, "members": [{"path": "config.json", + "size_bytes": 1, "kind": "git", "upstream_digest": "b" * 40, "sha256": "c" * 64}]} + value = {"schema_version": child.DISPATCH_SCHEMA, "attempt_ref": "grpo-diag-" + "d" * 24, + "source_commit": "e" * 40, "host_source_commit": "a" * 40, + "engine_gitlink": "e" * 40, "release": {}, "deployment_facts_digest": "f" * 64, + "image_id": "im-FIXTURE", "input_base64": __import__("base64").b64encode(raw).decode(), + "input_sha256": hashlib.sha256(raw).hexdigest(), "model": model, + "library_sources": child.pinned_library_sources()} + assert child.validate_dispatch(child.canonical(value))[1] == raw + for field, mutated in (("source_commit", "0" * 40), ("image_id", "bad"), + ("input_sha256", "0" * 64), ("library_sources", {})): + changed = dict(value, **{field: mutated}) + if field == "source_commit": + continue # A different valid commit needs the host's pushed-source check. + with pytest.raises(ValueError): + child.validate_dispatch(child.canonical(changed)) + changed = dict(value, library_sources={**value["library_sources"], + "unsloth/models/rl.py": "0" * 64}) + with pytest.raises(ValueError): + child.validate_dispatch(child.canonical(changed)) + + +def test_actual_core_constructor_only_has_no_train_or_generation_path(): + from Trainers.grpo.runtime_v1 import train_grpo_core + calls = [] + class Trainer: + def __init__(self, **kwargs): + calls.append(("construct", tuple(kwargs))) + def train(self, *args, **kwargs): + raise AssertionError("train called") + def generate(self, *args, **kwargs): + raise AssertionError("generate called") + def optimizer_step(self, *args, **kwargs): + raise AssertionError("optimizer called") + observer = lambda cls, kwargs: calls.append(("observe", tuple(kwargs))) + trainer, result = train_grpo_core(model=object(), tokenizer=object(), reward_fn=lambda: None, + formatted_dataset=object(), training_args=object(), callbacks=[], trainer_class=Trainer, + _constructor_only=True, _constructor_observer=observer) + assert isinstance(trainer, Trainer) and result is None + assert calls == [("observe", ("model", "processing_class", "reward_funcs", "args", "train_dataset", "callbacks")), + ("construct", ("model", "processing_class", "reward_funcs", "args", "train_dataset", "callbacks"))] + with pytest.raises(ValueError): + train_grpo_core(model=None, tokenizer=None, reward_fn=None, formatted_dataset=None, + training_args=None, callbacks=[], trainer_class=Trainer, _constructor_only=True) + + +def test_actual_run_grpo_constructor_only_preserves_reward_callbacks_and_stops(tmp_path, monkeypatch): + from Trainers.grpo import runtime_v1 as runtime + from Trainers.grpo.src import rewards, data_loader + recipe, dataset, plan, _ = _planned_input(tmp_path) + probe = verify_probe_input(derive_probe_input(recipe_path=recipe, plan_ready_path=plan, + dataset_path=dataset)) + workload = probe["workload"] + calls = [] + class Callback: + pass + class Trainer: + def __init__(self, **kwargs): + calls.append(("constructor", kwargs)) + def train(self, *args, **kwargs): + raise AssertionError("training path reached") + def generate(self, *args, **kwargs): + raise AssertionError("generation path reached") + def save_model(self, *args, **kwargs): + raise AssertionError("saving path reached") + class Loader: + @staticmethod + def load_model_and_tokenizer(*args, **kwargs): + calls.append(("model", kwargs)) + return object(), object(), False + @staticmethod + def apply_lora_adapters(**kwargs): + calls.append(("lora", kwargs)) + return object() + monkeypatch.setattr(runtime, "build_grpo_config", lambda *args, **kwargs: + SimpleNamespace(bf16=False, fp16=True)) + monkeypatch.setattr(runtime, "_QualificationMeasurements", lambda *args, **kwargs: + SimpleNamespace(reward_wrapper=lambda reward: reward, begin=lambda *a: None, + end=lambda *a: None, log=lambda *a: None)) + monkeypatch.setattr(rewards, "build_combined_reward_function", + lambda *args, **kwargs: (lambda *a: 1.0, {})) + monkeypatch.setattr(data_loader, "dataset_from_rows", lambda rows, **kwargs: rows) + monkeypatch.setattr(data_loader, "format_dataset_for_grpo", lambda rows, *args, **kwargs: rows) + model_root = tmp_path / "model" + output_root = tmp_path / "output" + model_root.mkdir() + output_root.mkdir() + trainer = runtime.run_grpo(config=workload["config"], dataset_bytes=dataset.read_bytes(), + reward_documents=workload["reward_documents"], reward_schema=workload["reward_schema"], + model_snapshot=model_root, model_identity=workload["model"], output_root=output_root, + workload=workload, metric_sink=lambda record: None, + _dependency_factory=lambda: {"trainer_class": Trainer, "config_class": object, + "bf16_supported": False, "model_loader": Loader, "dataset_class": object, + "callback_class": Callback}, + _packaged_artifacts=True, qualification_evidence=probe["qualification_policy"], + _child_deadline=time.monotonic() + 100, + _constructor_only=True, _constructor_observer=lambda cls, kwargs: + calls.append(("observer", tuple(kwargs)))) + assert isinstance(trainer, Trainer) + assert [name for name, _ in calls] == ["model", "lora", "observer", "constructor"] + assert calls[0][1]["dtype"] == "float16" + kwargs = calls[-1][1] + assert len(kwargs["callbacks"]) == 2 + assert all(isinstance(value, Callback) for value in kwargs["callbacks"]) + assert len(kwargs["train_dataset"]) == 4 + assert not list(output_root.iterdir()) + + +def test_fixed_region_projection_uses_class_code_and_verified_sources(): + class Base: + def __init__(self, *args, **kwargs): + pass + class Patched(Base): + def __init__(self, *args, **kwargs): + raise TypeError("never emitted") + class Wrapper(Patched): + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + try: + Wrapper() + except TypeError as error: + assert child._site(error, Wrapper, True) == "OTHER" + assert child._site(error, Wrapper, False) == "OTHER" + assert set(child._empty_result()) == set(child.validate_result(child.canonical(child._empty_result()))) + + +def test_isolated_entry_decodes_the_actual_four_row_dataset(tmp_path, monkeypatch): + from Trainers.grpo import runtime_v1 + recipe, dataset_path, plan, _ = _planned_input(tmp_path) + probe_raw = derive_probe_input(recipe_path=recipe, plan_ready_path=plan, + dataset_path=dataset_path) + probe = verify_probe_input(probe_raw) + snapshot = tmp_path / "snapshot" + snapshot.mkdir() + for name in ("state", "tmp", "tracking", "artifacts", "prepared_input", "cache"): + (tmp_path / name).mkdir() + paths = {name: str(tmp_path / name) for name in + ("prepared_input", "artifacts", "state", "tracking", "cache", "tmp")} + release = SimpleNamespace() + dispatch = {"release": {}, "model": {"ref": probe["workload"]["model"]["model_id"], + "revision": probe["workload"]["model"]["revision"], + "tokenizer_revision": probe["workload"]["model"]["tokenizer_revision"]}, + "library_sources": {}} + environment = tuple(sorted(os.environ.items())) + transport = child.canonical({"dispatch": dispatch, "private_manifest": {}, + "paths": paths, "environment": [list(item) for item in environment], + "deadline": time.monotonic() + 30}) + source = tmp_path / "transport.json" + source.write_bytes(transport) + output = tmp_path / "result.json" + fd = os.open(output, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) + monkeypatch.setattr(child, "validate_dispatch", lambda raw: (dispatch, probe_raw)) + monkeypatch.setattr(child, "parse_packaged_runtime_release", lambda raw: release) + monkeypatch.setattr(child.shared, "_inspect_release", lambda *a, **k: object()) + monkeypatch.setattr(child, "_installed_import_guard", lambda *a, **k: object()) + monkeypatch.setattr(child, "grpo_child_environment", lambda *a: environment) + monkeypatch.setattr(child.shared, "_retain_private_snapshot", lambda *a, **k: + SimpleNamespace(snapshot=snapshot, check=lambda: None, close=lambda: None)) + monkeypatch.setattr(child, "_source_verified", lambda sources: True) + monkeypatch.setattr(child, "_observation", lambda cls, kwargs, verified: {}) + monkeypatch.setattr(child, "sys", SimpleNamespace(flags=SimpleNamespace(isolated=1), + meta_path=sys.meta_path)) + monkeypatch.chdir(tmp_path / "tmp") + calls = [] + def run_grpo(**kwargs): + from Trainers.grpo.runtime_v1 import admit_grpo_inputs + _, rows, _, _ = admit_grpo_inputs(config=kwargs["config"], + dataset_bytes=kwargs["dataset_bytes"], + reward_documents=kwargs["reward_documents"], + reward_schema=kwargs["reward_schema"], + model_identity=kwargs["model_identity"], workload=kwargs["workload"]) + calls.append(len(rows)) + kwargs["_runtime_milestone"]("TRAINER_SETUP") + kwargs["_constructor_observer"](object, {}) + return object() + monkeypatch.setattr(runtime_v1, "run_grpo", run_grpo) + try: + child._isolated(source, hashlib.sha256(transport).hexdigest(), fd) + finally: + os.close(fd) + assert calls == [4] + assert child.validate_result(output.read_bytes())["status"] == "CONSTRUCTED" + + +def test_interrupt_reaps_actual_owned_child(tmp_path, monkeypatch): + original = child.subprocess.Popen + spawned = [] + class Interrupted: + def __init__(self, real): + self.real = real + self.pid = real.pid + self.returncode = None + self.once = True + def wait(self, timeout=None): + if self.once: + self.once = False + raise KeyboardInterrupt + self.returncode = self.real.wait(timeout=timeout) + return self.returncode + def launch(*args, **kwargs): + real = original(*args, **kwargs) + spawned.append(real) + return Interrupted(real) + monkeypatch.setattr(child.subprocess, "Popen", launch) + answer = tmp_path / "answer" + fd = os.open(answer, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) + try: + with pytest.raises(KeyboardInterrupt): + child._wait_isolated_child((sys.executable, "-c", "import time; time.sleep(60)"), + cwd=tmp_path, environment=dict(os.environ), result_fd=fd, + deadline=time.monotonic() + 10) + assert spawned[0].poll() is not None + finally: + if spawned and spawned[0].poll() is None: + os.killpg(spawned[0].pid, child.signal.SIGKILL) + spawned[0].wait(timeout=5) + os.close(fd) + + +def test_precision_distinction_matches_pinned_unsloth_branch_without_torch_import(monkeypatch): + class DType: + pass + half, bfloat, full = DType(), DType(), DType() + monkeypatch.setitem(sys.modules, "torch", SimpleNamespace(dtype=DType, + float16=half, bfloat16=bfloat)) + model = lambda dtype: SimpleNamespace(config=SimpleNamespace(dtype=dtype)) + args = SimpleNamespace(fp16=True, bf16=False) + assert child._dtype_relation(model(half), args) == "MATCH" + assert child._dtype_relation(model(bfloat), args) == "MISMATCH" + assert child._dtype_relation(model(full), args) == "MISMATCH" + assert child._dtype_relation(model(half), SimpleNamespace(fp16=False, bf16=True)) == "MISMATCH" + + +def test_actual_hung_and_flooding_cpu_children_are_reaped(tmp_path): + answer = tmp_path / "answer" + fd = os.open(answer, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o600) + try: + for code in ("import time; time.sleep(60)", + "import os,time\nwhile True: os.write(1,b'x'*65536)"): + start = time.monotonic() + with pytest.raises(TimeoutError): + child._wait_isolated_child((sys.executable, "-c", code), cwd=tmp_path, + environment=dict(os.environ), result_fd=fd, deadline=time.monotonic() + 0.2) + assert time.monotonic() - start < 3 + finally: + os.close(fd) + + +def test_result_file_has_one_aggregate_byte_cap_and_never_opens_line_logs(): + result = child.canonical(child._empty_result()) + class File: + def __init__(self, raw): self.raw = raw + def read(self, size): + assert size == 8193 + return self.raw[:size] + def close(self): self.closed = True + class Sandbox: + @property + def stdout(self): raise AssertionError("SDK by-line stdout must not be opened") + @property + def stderr(self): raise AssertionError("SDK by-line stderr must not be opened") + def __init__(self, raw): self.file = File(raw) + def open(self, path, mode): + assert path == child._RESULT_PATH and mode == "rb" + return self.file + accepted = Sandbox(result) + assert host._read_closed_result_file(accepted) == result + assert accepted.file.closed is True + oversized = Sandbox(b"x" * 8193) + with pytest.raises(ValueError): + host._read_closed_result_file(oversized) + assert oversized.file.closed is True + + +def test_bounded_hub_metadata_caps_unrelated_body_and_hung_read(): + class Response: + headers = {"Content-Encoding": "identity"} + def __init__(self, url, raw, *, hang=False): + self.url, self.raw, self.hang = url, raw, hang + def __enter__(self): return self + def __exit__(self, *args): pass + def geturl(self): return self.url + def read(self, size): + if self.hang: + time.sleep(60) + raw, self.raw = self.raw[:size], self.raw[size:] + return raw + class Opener: + def __init__(self, raw, *, hang=False): self.raw, self.hang = raw, hang + def open(self, request, timeout): + assert urlparse(request.full_url).hostname == "huggingface.co" + assert timeout <= 5 + return Response(request.full_url, self.raw, hang=self.hang) + url = "https://huggingface.co/api/models/fixture/tiny/revision/" + "a" * 40 + "?blobs=true" + with pytest.raises(ValueError): + host._bounded_hub_bytes(url, maximum=32, deadline=time.monotonic() + 2, + opener=Opener(b'{"unrelated":"' + b"x" * 100 + b'"}')) + start = time.monotonic() + with pytest.raises(TimeoutError): + host._bounded_hub_bytes(url, maximum=32, deadline=start + 0.15, + opener=Opener(b"{}", hang=True)) + assert time.monotonic() - start < 2 + + +def test_model_manifest_uses_pinned_git_blob_and_lfs_sha_without_weights(monkeypatch): + git = b'{"model_type":"fixture"}' + git_blob = hashlib.sha1(f"blob {len(git)}\0".encode() + git).hexdigest() + lfs_sha = "a" * 64 + revision = "b" * 40 + metadata = json.dumps({"sha": revision, "siblings": [ + {"rfilename": "config.json", "size": len(git), "blobId": git_blob}, + {"rfilename": "model.safetensors", "size": 100, "blobId": "c" * 40, + "lfs": {"size": 100, "sha256": lfs_sha, "pointerSize": 130}}, + ]}).encode() + urls = [] + def fake_read(url, *, maximum, deadline, opener): + urls.append(url) + return metadata if "/api/models/" in url else git + monkeypatch.setattr(host, "_bounded_hub_bytes", fake_read) + manifest = host.prepare_pinned_model_manifest("fixture/tiny", revision) + assert manifest["members"][0]["sha256"] == hashlib.sha256(git).hexdigest() + assert manifest["members"][1]["sha256"] == lfs_sha + assert len(urls) == 2 and "model.safetensors" not in urls[1] + + +def test_cleanup_requires_fresh_exact_id_terminal_proof(monkeypatch): + class Handle: + object_id = "sb-owned" + def terminate(self, *, wait): + self.requested = True + owned = Handle() + class SDK: + class Sandbox: + @staticmethod + def from_id(value, *, client): + assert value == "sb-owned" + return SimpleNamespace(object_id=value, poll=lambda: 137 if owned.__dict__.get("requested") else None) + assert host._cleanup_owned(SDK, object(), owned, expected_id="sb-owned", + deadline=time.monotonic() + 1) == "TERMINAL_CONFIRMED" + assert owned.requested is True + assert host._cleanup_owned(SDK, object(), owned, expected_id="sb-other", + deadline=time.monotonic() + 1) == "INDETERMINATE" + class FalseProofSDK: + class Sandbox: + @staticmethod + def from_id(value, *, client): + return SimpleNamespace(object_id="sb-other", poll=lambda: 0) + assert host._cleanup_owned(FalseProofSDK, object(), owned, expected_id="sb-owned", + deadline=time.monotonic() + 0.05) == "INDETERMINATE" + + +def test_admission_requires_independent_metadata_source_wheel_and_scope(monkeypatch, tmp_path): + from tuner.training.modal_host_runtime import ModalHostRuntimeV1 + recipe, dataset, plan, _ = _planned_input(tmp_path) + probe_raw = derive_probe_input(recipe_path=recipe, plan_ready_path=plan, dataset_path=dataset) + workload_model = verify_probe_input(probe_raw)["workload"]["model"] + content = b"{}" + model = {"ref": workload_model["model_id"], "revision": workload_model["revision"], + "tokenizer_revision": workload_model["tokenizer_revision"], + "members": [{"path": "config.json", "size_bytes": len(content), "kind": "git", + "upstream_digest": hashlib.sha1(b"blob 2\0" + content).hexdigest(), + "sha256": hashlib.sha256(content).hexdigest()}]} + metadata = SimpleNamespace(sha=model["revision"], siblings=[SimpleNamespace( + rfilename="config.json", size=2, lfs=None, + blob_id=model["members"][0]["upstream_digest"])]) + wheel = tmp_path / "source.whl" + wheel.write_bytes(b"wheel") + from tuner.runtime_profiles import load_runtime_profile + profiles_root = Path(__file__).resolve().parents[2] / "Trainers/runtime_profiles" + profile = load_runtime_profile("smollm2-135m-grpo-candidate-v1", profiles_root) + release = SimpleNamespace(worker_closure_digest="1" * 64, manifest_digest="2" * 64, + package_digest=hashlib.sha256(b"wheel").hexdigest(), image_ref=profile.image, + to_dict=lambda: {}) + deployment = SimpleNamespace(runtime_release_digest="2" * 64, image_id="im-example", + facts_digest="3" * 64, client_binding=SimpleNamespace(environment_ref="synaptic-smoke-v1")) + facts = SimpleNamespace(schema_version="synaptic-modal-packaged-runtime-facts/v2", + image_id="im-example", package_digest=release.package_digest, + worker_closure_digest="1" * 64, account_ref="synaptic-labs", + workspace_ref="synaptic-labs", environment_ref="synaptic-smoke-v1") + quote = SimpleNamespace(resource=SimpleNamespace(accelerator="L40S", accelerator_count=1, + timeout_seconds=120), maximum_cost_minor_units=100, + gpu_only_timeout_estimate_minor_units=80, quote_digest="4" * 64) + runtime = ModalHostRuntimeV1(release, deployment, facts, quote, + (("vo-model", "model-cache-fixture"),), b"a" * 32, None) + monkeypatch.setattr(host, "_source_pair", lambda engine, host_repo: ("e" * 40, "f" * 40)) + monkeypatch.setattr(host, "load_packaged_grpo_worker_closure", lambda: SimpleNamespace(digest="1" * 64)) + monkeypatch.setattr(host, "_verify_wheel_sources", lambda wheel, repository: "1" * 64) + monkeypatch.setattr(host, "_validate_deployment_release", lambda *args: None) + monkeypatch.setattr(host, "_model_role", lambda runtime: "vo-model") + monkeypatch.setattr(host, "validate_dispatch", lambda raw: None) + monkeypatch.setattr(host, "_pinned_model_info", lambda model: metadata) + args = dict(runtime=runtime, snapshot_runtime=runtime, + engine_repository=tmp_path, host_repository=tmp_path, wheel_path=wheel, + recipe_path=recipe, plan_ready_path=plan, dataset_path=dataset, + profiles_root=profiles_root, + input_bytes=probe_raw, model=model, + library_sources=child.pinned_library_sources(), + attempt_ref="grpo-diag-" + "b" * 24) + assert host.admit_modal_grpo_constructor(**args).image_id == "im-example" + for changed in ({"model": {**model, "members": [{**model["members"][0], "upstream_digest": "0" * 40}]}}, + {"runtime": ModalHostRuntimeV1(release, deployment, SimpleNamespace(**{**vars(facts), + "account_ref": "other"}), quote, (("vo-model", "model-cache-fixture"),), b"a" * 32, None)}): + with pytest.raises(host.ModalGRPOConstructorUnavailable): + host.admit_modal_grpo_constructor(**{**args, **changed}) + monkeypatch.setattr(host, "_pinned_model_info", lambda model: SimpleNamespace( + sha="0" * 40, siblings=metadata.siblings)) + with pytest.raises(host.ModalGRPOConstructorUnavailable): + host.admit_modal_grpo_constructor(**args) + monkeypatch.setattr(host, "_pinned_model_info", lambda model: metadata) + wheel.write_bytes(b"mutated") + with pytest.raises(host.ModalGRPOConstructorUnavailable): + host.admit_modal_grpo_constructor(**args) + + +def test_complete_grpo_wheel_source_comparison_rejects_one_changed_member(): + from io import BytesIO + from zipfile import ZipFile + from tuner.runtime.offline_grpo_worker import load_packaged_offline_grpo_worker_manifest + from tuner.runtime.packaged_grpo_worker_closure import load_packaged_grpo_worker_closure + root = Path(__file__).resolve().parents[2] + offline = load_packaged_offline_grpo_worker_manifest() + outer = load_packaged_grpo_worker_closure() + names = {item.path.as_posix() for item in offline.closure.members} + names.update(item["path"] for item in outer.members) + def wheel(changed=None): + output = BytesIO() + with ZipFile(output, "w") as archive: + for name in sorted(names): + raw = (root / name).read_bytes() + archive.writestr(name, raw + b"changed" if name == changed else raw) + return output.getvalue() + assert host._verify_wheel_sources(wheel(), root) == outer.digest + with pytest.raises(ValueError): + host._verify_wheel_sources(wheel("Trainers/grpo/runtime_v1.py"), root) + + +def test_distinct_receipt_retains_primary_and_cleanup_uncertainty(tmp_path): + from tuner.training.modal_host_storage import ModalHostStorageV1 + runtime = SimpleNamespace(qualification_key=b"x" * 32, + release=SimpleNamespace(manifest_digest="f" * 64), + deployment_facts=SimpleNamespace(facts_digest="1" * 64, image_id="im-fixture")) + admission = SimpleNamespace(attempt_ref="grpo-diag-" + "c" * 24, + dispatch_bytes=b"dispatch", source_commit="d" * 40, + host_source_commit="a" * 40, wheel_sha256="e" * 64, + runtime_release_digest="f" * 64, deployment_facts_digest="1" * 64, + image_id="im-fixture", snapshot_deployment_facts_digest="3" * 64, + model_cache_volume_id="vo-fixture", quote_digest="2" * 64) + receipt = host._receipt(runtime, admission, sandbox_id=None, + primary="INDETERMINATE", cleanup="INDETERMINATE", result=None, terminal=None, + failure_phase="CREATE") + assert receipt["body"]["cpu_qualified"] is False + assert receipt["body"]["training_qualified"] is False + assert receipt["body"]["diagnostic_type"] == "GRPO_CONSTRUCTOR_ONLY" + with ModalHostStorageV1(tmp_path / "journal.sqlite3", "diagnostic-test") as storage: + assert host._catalog(storage, "modal-grpo-constructor-results").publish_if_absent( + admission.attempt_ref, receipt) + assert host.read_modal_grpo_constructor_receipt(runtime, storage, admission.attempt_ref) == receipt + changed = dict(receipt, body={**receipt["body"], "cleanup": "TERMINAL_CONFIRMED"}) + assert changed != receipt + + +def test_restricted_modal_dispatch_uses_one_sandbox_and_separate_receipt(monkeypatch, tmp_path): + from tuner.training.modal_host_storage import ModalHostStorageV1 + from tuner.training.modal_host_runtime import ModalHostRuntimeV1 + recipe, dataset, plan, _ = _planned_input(tmp_path) + probe_raw = derive_probe_input(recipe_path=recipe, plan_ready_path=plan, dataset_path=dataset) + profiles_root = Path(__file__).resolve().parents[2] / "Trainers/runtime_profiles" + wheel = tmp_path / "source.whl" + wheel.write_bytes(b"wheel") + binding = SimpleNamespace(environment_ref="synaptic-smoke-v1") + release = SimpleNamespace(manifest_digest="1" * 64, worker_closure_digest="2" * 64, + package_digest=hashlib.sha256(b"wheel").hexdigest(), python_executable="/opt/python/bin/python") + deployment = SimpleNamespace(facts_digest="3" * 64, image_id="im-exact", + client_binding=binding) + facts = SimpleNamespace(app_name="app-exact", environment_ref="synaptic-smoke-v1") + quote = SimpleNamespace(resource=object(), maximum_cost_minor_units=100, + quote_digest="4" * 64) + runtime = ModalHostRuntimeV1(release, deployment, facts, quote, + (("vo-retained", "cache-exact"),), b"x" * 32, None) + source = SimpleNamespace(commit="5" * 40, host_commit="8" * 40) + monkeypatch.setattr(host, "_source_pair", lambda engine, host_repo: + (source.commit, source.host_commit)) + monkeypatch.setattr(host, "_verify_wheel_sources", lambda raw, repo: "2" * 64) + monkeypatch.setattr(host, "_model_role", lambda runtime: "vo-retained") + monkeypatch.setattr(host, "validate_dispatch", lambda raw: ({}, probe_raw)) + monkeypatch.setattr(host, "quote_modal_runtime_for_host", lambda **kwargs: quote) + monkeypatch.setattr(host, "_CurrentQualificationObserver", lambda **kwargs: object()) + monkeypatch.setattr(host, "_observe_current_layout", lambda *args: "observed") + result = child.canonical(child._empty_result()) + class Stdin: + def __init__(self): self.writes = [] + def write(self, raw): self.writes.append(raw) + def write_eof(self): pass + def drain(self): pass + class Sandbox: + object_id = "sb-exact" + def __init__(self): + self.stdin = Stdin() + self.acknowledged = False + def open(self, path, mode): + if path == child._READY_PATH and mode == "rb": + return SimpleNamespace(read=lambda size: b"1", close=lambda: None) + if path == child._RESULT_PATH and mode == "rb": + return SimpleNamespace(read=lambda size: result, close=lambda: None) + if path == child._ACK_PATH and mode == "xb": + def write(raw): + assert raw == b"1" + self.acknowledged = True + return SimpleNamespace(write=write, close=lambda: None) + raise AssertionError("unexpected sandbox file") + @property + def stdout(self): raise AssertionError("line-buffered logs forbidden") + @property + def stderr(self): raise AssertionError("line-buffered logs forbidden") + def terminate(self, *, wait): + raise AssertionError("completed Sandbox must not need termination") + sandbox = Sandbox() + calls = [] + class Volume: + object_id = "vo-retained" + _read_only = True + def hydrate(self, client): pass + def with_mount_options(self, *, read_only): + assert read_only is True + return self + class SDK: + __version__ = "1.5.4" + class App: + @staticmethod + def lookup(*args, **kwargs): return object() + class Volume: + @staticmethod + def from_name(name, **kwargs): + assert name == "cache-exact" and kwargs["create_if_missing"] is False + return Volume() + class Image: + @staticmethod + def from_id(value, *, client): return SimpleNamespace(object_id=value) + class Sandbox: + @staticmethod + def create(*args, **kwargs): + calls.append(kwargs) + return sandbox + @staticmethod + def from_id(value, *, client): + assert value == "sb-exact" + return SimpleNamespace(object_id=value, + poll=lambda: 0 if sandbox.acknowledged else None) + admission = host.ModalGRPOConstructorAdmission(b"dispatch", "grpo-diag-" + "6" * 24, + source.commit, source.host_commit, release.package_digest, release.manifest_digest, + deployment.facts_digest, + deployment.image_id, deployment.facts_digest, "vo-retained", "cache-exact", + quote.quote_digest, host._SEAL) + with ModalHostStorageV1(tmp_path / "journal.sqlite3", "diagnostic-test") as storage: + receipt = host.execute_modal_grpo_constructor_once(sdk=SDK, client=object(), + client_binding=binding, runtime=runtime, snapshot_runtime=runtime, + private_storage=storage, admission=admission, + engine_repository=tmp_path, host_repository=tmp_path, wheel_path=wheel, + recipe_path=recipe, plan_ready_path=plan, dataset_path=dataset, + profiles_root=profiles_root) + assert storage.attempts.resolve(admission.attempt_ref) is not None + assert receipt["body"]["primary"] == "COMPLETED" + assert receipt["body"]["cleanup"] == "TERMINAL_CONFIRMED" + assert receipt["body"]["cpu_qualified"] is False + assert receipt["body"]["training_qualified"] is False + assert len(calls) == 1 + assert calls[0]["gpu"] == "L40S" and calls[0]["cpu"] == 2.0 + assert calls[0]["memory"] == (8192, 8192) and calls[0]["timeout"] == 120 + assert calls[0]["block_network"] is True and calls[0]["secrets"] == [] + assert sandbox.stdin.writes == [b"dispatch"] + assert sandbox.acknowledged is True + with pytest.raises(Exception): + host.execute_modal_grpo_constructor_once(sdk=SDK, client=object(), + client_binding=binding, runtime=runtime, snapshot_runtime=runtime, + private_storage=storage, admission=admission, + engine_repository=tmp_path, host_repository=tmp_path, wheel_path=wheel, + recipe_path=recipe, plan_ready_path=plan, dataset_path=dataset, + profiles_root=profiles_root) + assert len(calls) == 1 + uncertain = host.ModalGRPOConstructorAdmission(b"dispatch", "grpo-diag-" + "7" * 24, + source.commit, source.host_commit, release.package_digest, + release.manifest_digest, deployment.facts_digest, + deployment.image_id, deployment.facts_digest, "vo-retained", "cache-exact", + quote.quote_digest, host._SEAL) + def fail_create(*args, **kwargs): + calls.append(kwargs) + raise TimeoutError + monkeypatch.setattr(SDK.Sandbox, "create", staticmethod(fail_create)) + uncertain_receipt = host.execute_modal_grpo_constructor_once(sdk=SDK, client=object(), + client_binding=binding, runtime=runtime, snapshot_runtime=runtime, + private_storage=storage, admission=uncertain, + engine_repository=tmp_path, host_repository=tmp_path, wheel_path=wheel, + recipe_path=recipe, plan_ready_path=plan, dataset_path=dataset, + profiles_root=profiles_root) + assert uncertain_receipt["body"]["primary"] == "INDETERMINATE" + assert uncertain_receipt["body"]["cleanup"] == "INDETERMINATE" + assert uncertain_receipt["body"]["failure_phase"] == "CREATE" + assert storage.attempts.resolve(uncertain.attempt_ref) is not None + assert len(calls) == 2 diff --git a/tests/execution/providers/test_modal_model_snapshot.py b/tests/execution/providers/test_modal_model_snapshot.py index 4cbf01f8..124773b4 100644 --- a/tests/execution/providers/test_modal_model_snapshot.py +++ b/tests/execution/providers/test_modal_model_snapshot.py @@ -109,7 +109,7 @@ def download(**kwargs): for name in ("persistent_root", "destination_root", "scratch_root") } for path in roots.values(): - path.mkdir() + path.mkdir(mode=0o700) return SimpleNamespace(files=files, info=info, calls=calls, roots=roots) @@ -226,9 +226,11 @@ def test_real_volume_binding_publishes_verified_model_fixture(fixture, monkeypat )) target_parent = tmp_path / "__modal" / "volumes" target = target_parent / "vo-test" - target.mkdir(parents=True) + (tmp_path / "__modal").mkdir(mode=0o700) + target_parent.mkdir(mode=0o700) + target.mkdir(mode=0o700) mount = tmp_path / "mnt" / "model-cache" - mount.parent.mkdir() + mount.parent.mkdir(mode=0o700) mount.symlink_to(target, target_is_directory=True) marker_name = ".synaptic-test-marker" marker = b"m" * 32 diff --git a/tuner/cloud/grpo_constructor_diagnostic.py b/tuner/cloud/grpo_constructor_diagnostic.py new file mode 100644 index 00000000..f05386d2 --- /dev/null +++ b/tuner/cloud/grpo_constructor_diagnostic.py @@ -0,0 +1,91 @@ +"""Provider-free GRPO constructor input and pinned model metadata admission. + +Execution belongs to the separate Modal runtime qualification boundary. This +module never imports the ML stack or downloads model files. +""" +from __future__ import annotations + +import base64 +import hashlib +import json +from pathlib import Path + +from tuner.cloud.derived_training_image import DerivedTrainingImageError, _read_object +from tuner.runtime.packaged_worker_closure import stable_read + + +INPUT_SCHEMA = "synaptic-grpo-constructor-probe-input/v3" + + +def _canonical(value: object) -> bytes: + return (json.dumps(value, ensure_ascii=True, sort_keys=True, + separators=(",", ":"), allow_nan=False) + "\n").encode("ascii") + + +def derive_probe_input(*, recipe_path: Path, plan_ready_path: Path, + dataset_path: Path, profiles_root: Path | None = None) -> bytes: + """Retain the normal compiler's four rows and the original JSONL bytes.""" + try: + plan_object, plan_raw = _read_object(plan_ready_path) + plan = plan_object["data"] + recipe, recipe_raw = _read_object(recipe_path) + if len(plan_raw) > 65536 or len(recipe_raw) > 65536: + raise ValueError + dataset_raw = stable_read(dataset_path, 65536) + from synaptic_tuner.api.v1.training_input import TrainingInputV1 + from synaptic_tuner.api.v1.training_sources import PreparedTrainingInputIdentity + from tuner.training.grpo_compilation import compile_grpo_input, admit_grpo_snapshot + + if (recipe["training_input"] != plan["training_input"] + or recipe["job"]["runtime_profile"] != plan["runtime_profile"] + or plan["resource_request"] != { + "accelerator": recipe["job"]["accelerator"], + "accelerator_count": recipe["job"]["accelerator_count"], + "timeout_seconds": recipe["job"]["timeout_seconds"]} + or plan["limits"] != recipe["limits"]): + raise ValueError + if profiles_root is not None: + from tuner.runtime_profiles import load_runtime_profile + profile = load_runtime_profile(plan["runtime_profile"], profiles_root) + if profile.profile_sha256 != plan["profile_sha256"]: + raise ValueError + model = plan["training_input"]["model"] + profile.resolve(model=model["ref"], model_revision=model["revision"], method="grpo") + training = TrainingInputV1.from_dict(plan["training_input"]) + prepared = PreparedTrainingInputIdentity.from_dict(plan["prepared_dataset"]) + compiled = compile_grpo_input(training, prepared) + _, rows, _, _ = admit_grpo_snapshot(training, prepared, dataset_raw) + if compiled.fingerprint != plan["core_workload_digest"] or len(rows) != 4: + raise ValueError + result = _canonical({ + "schema_version": INPUT_SCHEMA, + "workload": compiled.document, + "dataset_base64": base64.b64encode(dataset_raw).decode("ascii"), + "qualification_policy": recipe["limits"]["qualification_evidence"], + "row_count": len(rows), + "runtime_profile": plan["runtime_profile"], + "profile_sha256": plan["profile_sha256"], + "recipe_sha256": hashlib.sha256(recipe_raw).hexdigest(), + "plan_sha256": hashlib.sha256(plan_raw).hexdigest(), + "dataset_sha256": hashlib.sha256(dataset_raw).hexdigest(), + }) + if not 0 < len(result) <= 65536: + raise ValueError + return result + except (OSError, ValueError, KeyError, TypeError, json.JSONDecodeError, + DerivedTrainingImageError) as exc: + raise DerivedTrainingImageError("PROBE_INPUT_INVALID") from exc + + +def verify_probe_input(raw: bytes) -> dict[str, object]: + """Validate the exact projection before giving it any execution authority.""" + try: + from tuner.runtime.packaged_grpo_constructor_child import verify_probe_input as verify + return verify(raw) + except (ValueError, KeyError, TypeError, UnicodeError, json.JSONDecodeError) as exc: + raise DerivedTrainingImageError("PROBE_INPUT_INVALID") from exc + + +def probe_input_sha256(raw: bytes) -> str: + verify_probe_input(raw) + return hashlib.sha256(raw).hexdigest() diff --git a/tuner/execution/providers/hf_jobs_provider_v1/hf-parent-closure.json b/tuner/execution/providers/hf_jobs_provider_v1/hf-parent-closure.json index f01c4fc9..3eefedbf 100644 --- a/tuner/execution/providers/hf_jobs_provider_v1/hf-parent-closure.json +++ b/tuner/execution/providers/hf_jobs_provider_v1/hf-parent-closure.json @@ -1 +1 @@ -{"member_count":16,"members":[{"path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"path":"tuner/execution/providers/__init__.py","sha256":"8848731f5d1877c77ea8fcb66a82b85196aeafc06ba0e7463eca291ec1a5f41d","size_bytes":252},{"path":"tuner/execution/providers/contracts.py","sha256":"a21f22c8d373037d6426cd94dda48d7252438a08e115120188980dcaa34d6361","size_bytes":11506},{"path":"tuner/execution/providers/hf_jobs_provider_v1/__init__.py","sha256":"4394c79aea1de903fac6de6a5256eb45490ed4d4d0c6af8d06812e2d7c211524","size_bytes":107},{"path":"tuner/execution/providers/hf_jobs_provider_v1/bootstrap.py","sha256":"a9d7633131a521f5a928b5d15315c5c6617968b79d334197a19eff42f120f052","size_bytes":8595},{"path":"tuner/execution/providers/hf_jobs_provider_v1/source_worker.py","sha256":"d3ed18984aee25bc2bc73fffcb1ca24b05417e3ec961d73bda19b875adff2798","size_bytes":9539},{"path":"tuner/execution/providers/modal/__init__.py","sha256":"0f4e16bed9b4d05c59a5eef1d0ddc061720d5817a68a2bbe1073b20a19c27319","size_bytes":104},{"path":"tuner/execution/providers/modal/model_snapshot.py","sha256":"9d4762b821b2944a8a62932d0aec105c2f6920d64bf4ceed6907e5458799e337","size_bytes":14166},{"path":"tuner/execution/providers/modal/mounted_io.py","sha256":"e49da2e749d3be1ca19fdc7f581b29fb6d1e748a86fffcf0fde2be5cdc1a0d5b","size_bytes":13076},{"path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514}],"payload_bytes":117372,"schema_version":"synaptic-hf-parent-closure/v1"} \ No newline at end of file +{"member_count":16,"members":[{"path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"path":"tuner/execution/providers/__init__.py","sha256":"8848731f5d1877c77ea8fcb66a82b85196aeafc06ba0e7463eca291ec1a5f41d","size_bytes":252},{"path":"tuner/execution/providers/contracts.py","sha256":"a21f22c8d373037d6426cd94dda48d7252438a08e115120188980dcaa34d6361","size_bytes":11506},{"path":"tuner/execution/providers/hf_jobs_provider_v1/__init__.py","sha256":"4394c79aea1de903fac6de6a5256eb45490ed4d4d0c6af8d06812e2d7c211524","size_bytes":107},{"path":"tuner/execution/providers/hf_jobs_provider_v1/bootstrap.py","sha256":"a9d7633131a521f5a928b5d15315c5c6617968b79d334197a19eff42f120f052","size_bytes":8595},{"path":"tuner/execution/providers/hf_jobs_provider_v1/source_worker.py","sha256":"d3ed18984aee25bc2bc73fffcb1ca24b05417e3ec961d73bda19b875adff2798","size_bytes":9539},{"path":"tuner/execution/providers/modal/__init__.py","sha256":"0f4e16bed9b4d05c59a5eef1d0ddc061720d5817a68a2bbe1073b20a19c27319","size_bytes":104},{"path":"tuner/execution/providers/modal/model_snapshot.py","sha256":"0b6d665c347a65150bff21b8b21c19c1f437bbcd8db69821c4240f2c57e44cdf","size_bytes":14751},{"path":"tuner/execution/providers/modal/mounted_io.py","sha256":"e49da2e749d3be1ca19fdc7f581b29fb6d1e748a86fffcf0fde2be5cdc1a0d5b","size_bytes":13076},{"path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514}],"payload_bytes":117957,"schema_version":"synaptic-hf-parent-closure/v1"} \ No newline at end of file diff --git a/tuner/execution/providers/modal/inference-runtime.lock.json b/tuner/execution/providers/modal/inference-runtime.lock.json index d7d654c5..0d75a1f2 100644 --- a/tuner/execution/providers/modal/inference-runtime.lock.json +++ b/tuner/execution/providers/modal/inference-runtime.lock.json @@ -1 +1 @@ -{"base_registry_reference":"docker.io/vllm/vllm-openai@sha256:116aa00ee0b68855616a56e1d7e1ae937e591a8bd6969ee45cbcedb246ddf355","dependency_lock_path":"tuner/execution/providers/modal/inference-dependencies.lock","distributions":{"accelerate":"1.13.0","aiofile":"3.9.0","aiofiles":"25.1.0","aiohappyeyeballs":"2.6.1","aiohttp":"3.13.3","aiosignal":"1.4.0","annotated-doc":"0.0.4","annotated-types":"0.7.0","anthropic":"0.84.0","anyio":"4.12.1","apache-tvm-ffi":"0.1.9","astor":"0.8.1","attrs":"25.4.0","awscrt":"0.31.3","bitsandbytes":"0.49.2","blake3":"1.0.8","boto3":"1.42.65","botocore":"1.42.65","cachetools":"7.0.5","caio":"0.9.25","cbor2":"5.8.0","certifi":"2026.2.25","cffi":"2.0.0","charset-normalizer":"3.4.5","click":"8.3.1","cloudpickle":"3.1.2","compressed-tensors":"0.13.0","cryptography":"46.0.5","cuda-bindings":"12.9.4","cuda-pathfinder":"1.4.2","cuda-python":"12.9.4","cufile-python":"0.2.0","cupy-cuda12x":"14.0.1","deep-ep":"1.2.1+73b6ea4","deep-gemm":"2.3.0+477618c","depyf":"0.20.0","dill":"0.4.1","diskcache":"5.6.3","distro":"1.9.0","dnspython":"2.8.0","docstring-parser":"0.17.0","einops":"0.8.2","email-validator":"2.3.0","fastapi":"0.135.1","fastapi-cli":"0.0.24","fastapi-cloud-cli":"0.14.1","fastar":"0.8.0","filelock":"3.25.1","flashinfer-cubin":"0.6.4","flashinfer-jit-cache":"0.6.4+cu129","flashinfer-python":"0.6.4","frozenlist":"1.8.0","fsspec":"2026.2.0","gguf":"0.18.0","google-api-core":"2.30.0","google-auth":"2.49.0","google-cloud-core":"2.5.0","google-cloud-storage":"3.9.0","google-crc32c":"1.8.0","google-resumable-media":"2.8.0","googleapis-common-protos":"1.73.0","grpcio":"1.78.0","grpcio-reflection":"1.78.0","grpclib":"0.4.9","h11":"0.16.0","h2":"4.4.1","hf-transfer":"0.1.9","hf-xet":"1.3.2","hpack":"4.2.0","httpcore":"1.0.9","httptools":"0.7.1","httpx":"0.28.1","httpx-sse":"0.4.3","huggingface-hub":"0.36.2","humanize":"4.15.0","hyperframe":"6.1.0","idna":"3.11","ijson":"3.5.0","importlib-metadata":"8.7.1","interegular":"0.3.3","jinja2":"3.1.6","jiter":"0.13.0","jmespath":"1.1.0","jsonschema":"4.26.0","jsonschema-specifications":"2025.9.1","kaldi-native-fbank":"1.22.3","lark":"1.2.2","llguidance":"1.3.0","llvmlite":"0.44.0","lm-format-enforcer":"0.11.3","lmcache":"0.4.0","loguru":"0.7.3","markdown-it-py":"4.0.0","markupsafe":"3.0.3","mcp":"1.26.0","mdurl":"0.1.2","mistral-common":"1.9.1","modal":"1.5.4","model-hosting-container-standards":"0.1.13","modelscope":"1.34.0","mooncake-transfer-engine":"0.3.9","mpmath":"1.3.0","msgpack":"1.1.2","msgspec":"0.20.0","multidict":"6.7.1","networkx":"3.6.1","ninja":"1.13.0","nixl":"0.9.0","nixl-cu12":"0.10.1","numba":"0.61.2","numpy":"2.2.6","nvidia-cublas-cu12":"12.9.1.4","nvidia-cuda-cupti-cu12":"12.9.79","nvidia-cuda-nvrtc-cu12":"12.9.86","nvidia-cuda-runtime-cu12":"12.9.79","nvidia-cudnn-cu12":"9.10.2.21","nvidia-cudnn-frontend":"1.18.0","nvidia-cufft-cu12":"11.4.1.4","nvidia-cufile-cu12":"1.14.1.1","nvidia-curand-cu12":"10.3.10.19","nvidia-cusolver-cu12":"11.7.5.82","nvidia-cusparse-cu12":"12.5.10.65","nvidia-cusparselt-cu12":"0.7.1","nvidia-cutlass-dsl":"4.4.1","nvidia-cutlass-dsl-libs-base":"4.4.1","nvidia-ml-py":"13.590.48","nvidia-nccl-cu12":"2.27.5","nvidia-nvjitlink-cu12":"12.9.86","nvidia-nvshmem-cu12":"3.4.5","nvidia-nvtx-cu12":"12.9.79","nvtx":"0.2.14","openai":"2.24.0","openai-harmony":"0.0.8","opencv-python-headless":"4.13.0.92","opentelemetry-api":"1.40.0","opentelemetry-exporter-otlp":"1.40.0","opentelemetry-exporter-otlp-proto-common":"1.40.0","opentelemetry-exporter-otlp-proto-grpc":"1.40.0","opentelemetry-exporter-otlp-proto-http":"1.40.0","opentelemetry-proto":"1.40.0","opentelemetry-sdk":"1.40.0","opentelemetry-semantic-conventions":"0.61b0","opentelemetry-semantic-conventions-ai":"0.4.15","outlines-core":"0.2.11","packaging":"26.0","partial-json-parser":"0.2.1.1.post7","pillow":"12.1.1","pip":"26.0.1","prometheus-client":"0.24.1","prometheus-fastapi-instrumentator":"7.1.0","propcache":"0.4.1","proto-plus":"1.27.1","protobuf":"6.33.5","psutil":"7.2.2","py-cpuinfo":"9.0.0","pyasn1":"0.6.2","pyasn1-modules":"0.4.2","pybase64":"1.4.3","pycountry":"26.2.16","pycparser":"3.0","pydantic":"2.12.5","pydantic-core":"2.41.5","pydantic-extra-types":"2.11.0","pydantic-settings":"2.13.1","pygments":"2.19.2","pyjwt":"2.11.0","python-dateutil":"2.9.0.post0","python-dotenv":"1.2.2","python-json-logger":"4.0.0","python-multipart":"0.0.22","pyyaml":"6.0.3","pyzmq":"27.1.0","quack-kernels":"0.3.2","ray":"2.54.0","redis":"7.3.0","referencing":"0.37.0","regex":"2026.2.28","requests":"2.32.5","rich":"14.3.3","rich-toolkit":"0.19.7","rignore":"0.7.6","rpds-py":"0.30.0","rsa":"4.9.1","runai-model-streamer":"0.15.7","runai-model-streamer-gcs":"0.15.7","runai-model-streamer-s3":"0.15.7","s3transfer":"0.16.0","safetensors":"0.7.0","sentencepiece":"0.2.1","sentry-sdk":"2.54.0","setproctitle":"1.3.7","setuptools":"80.10.2","setuptools-scm":"9.2.2","shellingham":"1.5.4","six":"1.17.0","sniffio":"1.3.1","sortedcontainers":"2.4.0","sse-starlette":"3.3.2","starlette":"0.52.1","supervisor":"4.3.0","sympy":"1.14.0","synaptic-tuner":"1.1.0","synchronicity":"0.12.5","tabulate":"0.10.0","tiktoken":"0.12.0","timm":"1.0.25","tokenizers":"0.22.2","toml":"0.10.2","torch":"2.10.0+cu129","torch-c-dlpack-ext":"0.1.5","torchaudio":"2.10.0+cu129","torchvision":"0.25.0+cu129","tqdm":"4.67.3","transformers":"4.57.6","triton":"3.6.0","typer":"0.24.1","types-certifi":"2021.10.8.3","types-toml":"0.10.8.20260518","typing-extensions":"4.15.0","typing-inspection":"0.4.2","urllib3":"2.6.3","uv":"0.10.9","uvicorn":"0.41.0","uvloop":"0.22.1","vllm":"0.17.1","watchfiles":"1.1.1","websockets":"16.0","xgrammar":"0.1.29","yarl":"1.23.0","zipp":"3.23.0"},"python":{"executable":"/opt/synaptic-inference/bin/python","executable_sha256":"185be5c01624944b5c422a8d1e4995a7df010036a99c9f9d84a80a09ae1bd72d","implementation":"cpython","version":"3.12.13"},"schema_version":"synaptic-modal-inference-runtime-lock/v1","sdk_version":"1.5.4","source_inventory":[{"path":"Evaluator/__init__.py","sha256":"d1b5aea793a95c47618d4b2469a8bae0b14fd11f81501c1d71546469a39a5cbd","size_bytes":3061},{"path":"Evaluator/base_client.py","sha256":"ac7e77909bdf161a4fd9b1ad2e1e2cfe085ec31f36baa6b19f8976f7b2358cbd","size_bytes":19362},{"path":"Evaluator/config.py","sha256":"94fbe8e198d065141ff9ad002b0a6e26e56d865fc1e3a3fb23479a8e69d09303","size_bytes":21956},{"path":"Evaluator/enums.py","sha256":"2ced9426d860e9ec410e376d9cf853d75e8e111e78dd0bee31e25bee295e4854","size_bytes":3371},{"path":"Evaluator/openai_compat_client.py","sha256":"6ac6211185d2c57d3b31022d31478e3253046fd6cb4de1903d8d0b209bad3a79","size_bytes":5174},{"path":"Evaluator/protocols.py","sha256":"239fe049b5fe4c9e9861aede487feac1f06a7642891378a1ccfba444072ad706","size_bytes":4560},{"path":"Evaluator/verified_vllm_chat.py","sha256":"f130e47d661aa77d68c104084fafe9e5df3af88684ef5a3139f4d53ff2e32465","size_bytes":5916},{"path":"Evaluator/vllm_client.py","sha256":"4e88d2055dc1b32cc22a1393cdf038ccd76d91d915a8be0774b0a8ec87bd4d1e","size_bytes":1995},{"path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"path":"synaptic_tuner/api/v1/planning.py","sha256":"e715b105a513fa56cfa510416e725471f8f3100cb34257be74bef3e40ca78460","size_bytes":9569},{"path":"synaptic_tuner/api/v1/providers.py","sha256":"eca5efd5a6dee453543ae338ffddf9451d34ce1743e4ab4bde024b47be571ec0","size_bytes":4940},{"path":"synaptic_tuner/api/v1/results.py","sha256":"65e530325c92c7afa18e3ca5f2ad9a11baf7d590221f08c33badfbf2d5f2c7b2","size_bytes":2236},{"path":"synaptic_tuner/api/v1/runs_facade.py","sha256":"bf6155726c99cfc5574385009e20f4f65b69a9ad4b96e33cf7065263d1d4c455","size_bytes":23572},{"path":"synaptic_tuner/api/v1/training_facade.py","sha256":"bf094d0e152f5861bf8a8ff51962e0700123d9b43ed91d914b9e78bbea6facd9","size_bytes":11118},{"path":"synaptic_tuner/api/v1/training_input.py","sha256":"209326325f8796ab7d038878e92def2c5521d7d98b7b9cc4dd1766d2e4f68c9d","size_bytes":38650},{"path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"path":"tuner/cloud/runtime_layout.py","sha256":"2a4cd1f5fc63d95bb3f0b5bc4f44e616bb0308d9509aceb198943953746ff705","size_bytes":9435},{"path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"path":"tuner/execution/_effect_executor.py","sha256":"7aa3c523d971486417fe735c62a5cb3fe7e0698caa514d0da047293ad6b3e9cd","size_bytes":500},{"path":"tuner/execution/broker.py","sha256":"c504bd734b41ef9785fe5fa591966bb0b0bce3b43dcce0695b4ee0c142ca3d1e","size_bytes":6305},{"path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"path":"tuner/execution/coordinator_v1/__init__.py","sha256":"453718a1827cfb23e7ca0c908d3d256f62ca3e3fff8f75ed8a785f8336b17bbe","size_bytes":66},{"path":"tuner/execution/coordinator_v1/coordinator.py","sha256":"9bc8ecb60d472ad0b3c0d3192d8c73d88eb28dc43af55677fbdd4b3673b71ef9","size_bytes":47018},{"path":"tuner/execution/coordinator_v1/cursors.py","sha256":"5f7c899064dd26959aeed6b328b236b01f7ff549e2d29348bd45dc0480ede26b","size_bytes":9088},{"path":"tuner/execution/coordinator_v1/foundation.py","sha256":"4433c390075544df5516a5e213af4b4b697485ff7627f5a6dcace21a98029f94","size_bytes":12972},{"path":"tuner/execution/coordinator_v1/model.py","sha256":"1d2150d588f6b84f1b58913df398365a9491d6486a8a871bb2a22fa98cafc722","size_bytes":75837},{"path":"tuner/execution/coordinator_v1/ports.py","sha256":"7a028c32c3a3eab905bbc202248b4bf39411cd22e73fddacf5099b7d15dc3832","size_bytes":9343},{"path":"tuner/execution/coordinator_v1/state_machine.py","sha256":"e5e63db3a85fb9fc5b1d7bb5444691f88f308cfb0428842c0801b06d6f2fbb3a","size_bytes":50176},{"path":"tuner/execution/coordinator_v1/stores.py","sha256":"3a875f9e9d7d7d9899d5deba2f4d68cfada12bea499d5897592c9383ba2b16dc","size_bytes":39045},{"path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"path":"tuner/execution/foundation_v2/__init__.py","sha256":"1651bf897879603a83e7824f60d369d52ac5d0308c96f988c24465f33112cd77","size_bytes":77},{"path":"tuner/execution/foundation_v2/authority.py","sha256":"a0eb5b3537a59c7b4d33e9b8617abd072376545d808f1989f562704dfa1ad024","size_bytes":9958},{"path":"tuner/execution/foundation_v2/canonical.py","sha256":"80620a6088d137bfacb210e000c406632bf569de66350628fb0e0aee612babfb","size_bytes":4047},{"path":"tuner/execution/foundation_v2/commands.py","sha256":"0fff7f05401094a60a390514622bbaa5649a8fb1b5f85f938e856192dac34e5c","size_bytes":8891},{"path":"tuner/execution/foundation_v2/executors.py","sha256":"27a428368c7aa4b9b502ae8028de03b48b817ea9d237a08dd112e7a6afc20454","size_bytes":5331},{"path":"tuner/execution/foundation_v2/identities.py","sha256":"720ae3938627f075c181eb5a125d118df87c9d6f7f24869107c74df9fbc44388","size_bytes":2794},{"path":"tuner/execution/foundation_v2/observations.py","sha256":"3642e4fd35478be5cb9a41df8f758ec4581d9f341f470bad1a4a2fbf8093636c","size_bytes":2566},{"path":"tuner/execution/foundation_v2/operations.py","sha256":"0f3c12d916a71a1fe20017b5ff7e99676c0acbf8bae7fac903d629c1aa3a9e6d","size_bytes":2259},{"path":"tuner/execution/foundation_v2/preparation.py","sha256":"09883f2455e521a7baa9b30fbb81518aa33ab2845392009d87cb54606ff80f1d","size_bytes":3575},{"path":"tuner/execution/foundation_v2/receipts.py","sha256":"c327476257236209ea1e29258fa55d90859bb262d27c132870d21399dfb5f626","size_bytes":15339},{"path":"tuner/execution/foundation_v2/references.py","sha256":"53d8e2070395a4c7f5c5a9361532f7a78de714003c760b55a23366394561b37f","size_bytes":3336},{"path":"tuner/execution/foundation_v2/repository.py","sha256":"d311b21fe8f2ed4b85d50ffa55fca370fe6452841d7a0189c5d56b841f69ef7f","size_bytes":57985},{"path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"path":"tuner/execution/operation.py","sha256":"9f86c585fc7ac3f7b2689a4701d598e32a382c3d1e0b87a7732ef35235568504","size_bytes":13702},{"path":"tuner/execution/providers/__init__.py","sha256":"8848731f5d1877c77ea8fcb66a82b85196aeafc06ba0e7463eca291ec1a5f41d","size_bytes":252},{"path":"tuner/execution/providers/contracts.py","sha256":"a21f22c8d373037d6426cd94dda48d7252438a08e115120188980dcaa34d6361","size_bytes":11506},{"path":"tuner/execution/providers/modal/__init__.py","sha256":"0f4e16bed9b4d05c59a5eef1d0ddc061720d5817a68a2bbe1073b20a19c27319","size_bytes":104},{"path":"tuner/execution/providers/modal/binding.py","sha256":"2d2acb7685798f487988b7690c1dbaf6ddc49e34ed8cd3600add438e17fc9ad5","size_bytes":1810},{"path":"tuner/execution/providers/modal/config.py","sha256":"bfa32ed26f3b8ece5e89e1a3524cb81e85ae3240e72ce93430ffb82dbb02f525","size_bytes":20788},{"path":"tuner/execution/providers/modal/contracts.py","sha256":"249e75a9ad13b93c65ac95cb100dc4fe24ed4c59674770832af3d5a9c4112dd2","size_bytes":6449},{"path":"tuner/execution/providers/modal/control.py","sha256":"19e05126196be5009c2cedac36a39008e82ca0bd17c9ec461f68f6ef26ece750","size_bytes":10359},{"path":"tuner/execution/providers/modal/coordinator_adapter.py","sha256":"619ba5e4e10f1a26ad4c154490da9328f9863581df02bf768e48e045ba46e413","size_bytes":13178},{"path":"tuner/execution/providers/modal/coordinator_binding.py","sha256":"e9267c37dce7273ffcad5f1db5332e205ba5282a9fbad4a647581e23c08b821e","size_bytes":4125},{"path":"tuner/execution/providers/modal/coordinator_bundle.py","sha256":"5d8a10d2c4a0dce83174eeca202c377730820ced6945be6f7717e3a0cfd91eb0","size_bytes":31316},{"path":"tuner/execution/providers/modal/coordinator_dispatch.py","sha256":"8e5bc475c6b7d15345e22e1f8cebdabcffab5a01ffefb7d4acbfb073be02b49f","size_bytes":10571},{"path":"tuner/execution/providers/modal/coordinator_launch.py","sha256":"5975bee2d73dd09e02008ede368ebc8f03325815e8bcfd4c093736214d5f0956","size_bytes":10982},{"path":"tuner/execution/providers/modal/coordinator_preflight.py","sha256":"f452e906d37fa9dba932e9df46f829447c95ec276191e34ec3e46831c89da875","size_bytes":13746},{"path":"tuner/execution/providers/modal/coordinator_reader.py","sha256":"92a1c21725587a88435d30b6b687f63711a12ad4c77345f0776521d6503103f4","size_bytes":28524},{"path":"tuner/execution/providers/modal/coordinator_staging.py","sha256":"de47b688e859ab96a612c01b1c7869d814e40b83ae81dcd7f2b1132e18c794f0","size_bytes":20880},{"path":"tuner/execution/providers/modal/coordinator_submit_preparation.py","sha256":"28814779c7f086016ae3ff617e0f1d4cbd8b52f518d3d889a905ea03d70456ec","size_bytes":5213},{"path":"tuner/execution/providers/modal/coordinator_wire.py","sha256":"07d46b92b33e30cb45bb6b1139ce4f80afdb09e1a17a3f3c2526ddcfae6983c9","size_bytes":12998},{"path":"tuner/execution/providers/modal/deployment_identity.py","sha256":"1b7007e5a9c11abdfe0ad4f106f742faa0ee556921b4017d3b48cb772bd0da4f","size_bytes":1196},{"path":"tuner/execution/providers/modal/deployment_v1.py","sha256":"579ace8adbddfe30e09d5a39df9926a54b111a139c704fc7efc7cebf533acf94","size_bytes":3628},{"path":"tuner/execution/providers/modal/facade.py","sha256":"55b89b3d0faff6238eeead068d6963d26f27749d725d2f2b8ae32449b348e32d","size_bytes":17704},{"path":"tuner/execution/providers/modal/inference-dependencies.lock","sha256":"ca4e3f0d7fa06533ad1127d84b539279791e37f07dc62cc81fbf0edd3a59de88","size_bytes":8458},{"path":"tuner/execution/providers/modal/inference-worker-closure.json","sha256":"cf21d2b602504a7ccccd907bf8358d485501dc6798eae8a0779b70de5c305fd6","size_bytes":18118},{"path":"tuner/execution/providers/modal/inference_artifacts.py","sha256":"5816f0c5662de8edad752554ba149d9527ced40ebdceeccc993c136ab099bff3","size_bytes":14973},{"path":"tuner/execution/providers/modal/inference_binding.py","sha256":"f2e8c4f1d8fc1f5e70702c88cbfff044a1f4f569e06b9a31c33a0cb9aaaf5894","size_bytes":23363},{"path":"tuner/execution/providers/modal/inference_bootstrap.py","sha256":"db2f09c1dfb2aee667b46fbb6566be8b2922d8e762896e5577d5c5078ad7d122","size_bytes":12546},{"path":"tuner/execution/providers/modal/inference_channel.py","sha256":"23492b4c7f48d86d23fb21ffd3902ac3f48bae381c30ea4759c803577911592e","size_bytes":13438},{"path":"tuner/execution/providers/modal/inference_commands.py","sha256":"1fa2dd667cb3b33e012d37b0688bfe08fc7a7cec52230a202f1f9ddda70110c5","size_bytes":4977},{"path":"tuner/execution/providers/modal/inference_entrypoint.py","sha256":"6ec8bdcbbf6d87fb5fc02a2c42d2c0f344ea965ccc4c0d94a630055c6e779991","size_bytes":10873},{"path":"tuner/execution/providers/modal/inference_model.py","sha256":"c08e1a35bb71a11c3466aedfbd8d3de87cf32d6cbd85820c3dcde2fefd14fd65","size_bytes":5935},{"path":"tuner/execution/providers/modal/inference_preparation.py","sha256":"43a1c883a8e5b2f25364e3a20f59d92a852af15fda0c9d506c213c0bf185fc32","size_bytes":49436},{"path":"tuner/execution/providers/modal/inference_runtime.py","sha256":"f8ba0d5744f90ca95e1fea3e5faf293e153a91047ffb09596f0305e1339d56ee","size_bytes":19283},{"path":"tuner/execution/providers/modal/inference_wire.py","sha256":"0492c10c6a384596bba04b3b409dc34f1f1a4afa88b7321d0d1a6fac5e29a1fb","size_bytes":20150},{"path":"tuner/execution/providers/modal/inference_worker.py","sha256":"cfb27e938c60757ef08c80b918fb522012c84918e7a0cd41a63de2cb2486cb87","size_bytes":15657},{"path":"tuner/execution/providers/modal/inference_workload.py","sha256":"4eaf399485e76829a98f57907c012658ce41d02745978827f21262ce2be94ff8","size_bytes":20421},{"path":"tuner/execution/providers/modal/manifest.py","sha256":"fc2ab4dcc46d136a681bf124d7a9fc3e5deedeeb3641d798fd207cdb62bb506f","size_bytes":8579},{"path":"tuner/execution/providers/modal/model_snapshot.py","sha256":"9d4762b821b2944a8a62932d0aec105c2f6920d64bf4ceed6907e5458799e337","size_bytes":14166},{"path":"tuner/execution/providers/modal/mounted_io.py","sha256":"e49da2e749d3be1ca19fdc7f581b29fb6d1e748a86fffcf0fde2be5cdc1a0d5b","size_bytes":13076},{"path":"tuner/execution/providers/modal/prepared_input.py","sha256":"c16cef5cfcc73ab9a7826977b26e18912ac46409d91b9eae165893ed265896ac","size_bytes":8597},{"path":"tuner/execution/providers/modal/resolution.py","sha256":"5c7268aa644103ae0712b03ed55918527d6e2ed6f4120ef169866b0ffd1cbf6f","size_bytes":30179},{"path":"tuner/execution/providers/modal/runtime.py","sha256":"ada13733021a6dab19331da80076df630055abc5bddc192a34a5cda65ac718c4","size_bytes":14810},{"path":"tuner/execution/providers/modal/worker_ports.py","sha256":"adf334368dbb75e40177078cd32aedc445cd9dde8b5ece2401319d472dbccdb0","size_bytes":2928},{"path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"path":"tuner/inference/__init__.py","sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","size_bytes":0},{"path":"tuner/inference/chat_session.py","sha256":"b95a829f834406a0a0c152f9af713225fe205a9225ad7b0f959b1ba801dbcac2","size_bytes":22355},{"path":"tuner/inference/owned_process.py","sha256":"f4cbb4e1c2305615b2c127e1103f60b1e17e955fe20a2a847b1d88b7ba8c4c98","size_bytes":8832},{"path":"tuner/inference/retrieved_model.py","sha256":"4a8cf2b2cbe4f03bf7c4d240c5e59a6ec6d20826c686b3a06c0bc1642fa234d8","size_bytes":31703},{"path":"tuner/inference/run_chat.py","sha256":"9263d325224bee4b9a0e7c16045e30766d9a2e2d2b550f0508040161687d3cfc","size_bytes":6228},{"path":"tuner/inference/serving_target.py","sha256":"50ec8054d1d6d08724d7a29183ca54a04d9c915d2c04274bdac6834764bdb800","size_bytes":12227},{"path":"tuner/inference/vllm_runtime.py","sha256":"fafbcfdeee78143a2760e5fbf29621897e9be014814eefb69f1167b188c296c1","size_bytes":26019},{"path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"path":"tuner/runtime/dispatch.py","sha256":"4c41faf87d7e620bdc281490bf3725550b83dacca8b15ea47bab253f669cdd8e","size_bytes":35463},{"path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"path":"tuner/runtime/verification.py","sha256":"4c2314b20cbf7937a3f24aa90676000b9e21f0ab6701b2c1225a53e5596bf4a0","size_bytes":53856},{"path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"path":"tuner/training/contracts.py","sha256":"91c4d66d9810cc0471981c3d4ef4b54feaed5b92eb41a468f31201f8111303b5","size_bytes":21504},{"path":"tuner/training/coordinator_material.py","sha256":"c563aa3b1fbd940ddd3b8e56e4f33de70675cb831e938175f40debc0645fd9c7","size_bytes":12950},{"path":"tuner/training/grpo_compilation.py","sha256":"c6268486375bbc16f1888bb43d07b93a8de280d000a774914d59d7fd05ca5cc0","size_bytes":8998},{"path":"tuner/training/methods/__init__.py","sha256":"99db689246273707ae3df5087859ebb510e84b664f9f173c0d5827abc3e6a3ea","size_bytes":361},{"path":"tuner/training/methods/sft.py","sha256":"b501f9c214dec4b0b1550543281c25aba8dbb102604df253c641c85f7cae587b","size_bytes":9997},{"path":"tuner/training/packaged_compilation.py","sha256":"477caede6902fc587af680889b93fd9e0e42493d1a6653f3cd7413a9c7cadde8","size_bytes":9413},{"path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722},{"path":"tuner/training/post_training.py","sha256":"ba8c2ea3fe49a647d502c6a874a44658cc2e575d4cd5729966102b0ebb973a88","size_bytes":7971},{"path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"worker_closure_manifest_path":"tuner/execution/providers/modal/inference-worker-closure.json"} +{"base_registry_reference":"docker.io/vllm/vllm-openai@sha256:116aa00ee0b68855616a56e1d7e1ae937e591a8bd6969ee45cbcedb246ddf355","dependency_lock_path":"tuner/execution/providers/modal/inference-dependencies.lock","distributions":{"accelerate":"1.13.0","aiofile":"3.9.0","aiofiles":"25.1.0","aiohappyeyeballs":"2.6.1","aiohttp":"3.13.3","aiosignal":"1.4.0","annotated-doc":"0.0.4","annotated-types":"0.7.0","anthropic":"0.84.0","anyio":"4.12.1","apache-tvm-ffi":"0.1.9","astor":"0.8.1","attrs":"25.4.0","awscrt":"0.31.3","bitsandbytes":"0.49.2","blake3":"1.0.8","boto3":"1.42.65","botocore":"1.42.65","cachetools":"7.0.5","caio":"0.9.25","cbor2":"5.8.0","certifi":"2026.2.25","cffi":"2.0.0","charset-normalizer":"3.4.5","click":"8.3.1","cloudpickle":"3.1.2","compressed-tensors":"0.13.0","cryptography":"46.0.5","cuda-bindings":"12.9.4","cuda-pathfinder":"1.4.2","cuda-python":"12.9.4","cufile-python":"0.2.0","cupy-cuda12x":"14.0.1","deep-ep":"1.2.1+73b6ea4","deep-gemm":"2.3.0+477618c","depyf":"0.20.0","dill":"0.4.1","diskcache":"5.6.3","distro":"1.9.0","dnspython":"2.8.0","docstring-parser":"0.17.0","einops":"0.8.2","email-validator":"2.3.0","fastapi":"0.135.1","fastapi-cli":"0.0.24","fastapi-cloud-cli":"0.14.1","fastar":"0.8.0","filelock":"3.25.1","flashinfer-cubin":"0.6.4","flashinfer-jit-cache":"0.6.4+cu129","flashinfer-python":"0.6.4","frozenlist":"1.8.0","fsspec":"2026.2.0","gguf":"0.18.0","google-api-core":"2.30.0","google-auth":"2.49.0","google-cloud-core":"2.5.0","google-cloud-storage":"3.9.0","google-crc32c":"1.8.0","google-resumable-media":"2.8.0","googleapis-common-protos":"1.73.0","grpcio":"1.78.0","grpcio-reflection":"1.78.0","grpclib":"0.4.9","h11":"0.16.0","h2":"4.4.1","hf-transfer":"0.1.9","hf-xet":"1.3.2","hpack":"4.2.0","httpcore":"1.0.9","httptools":"0.7.1","httpx":"0.28.1","httpx-sse":"0.4.3","huggingface-hub":"0.36.2","humanize":"4.15.0","hyperframe":"6.1.0","idna":"3.11","ijson":"3.5.0","importlib-metadata":"8.7.1","interegular":"0.3.3","jinja2":"3.1.6","jiter":"0.13.0","jmespath":"1.1.0","jsonschema":"4.26.0","jsonschema-specifications":"2025.9.1","kaldi-native-fbank":"1.22.3","lark":"1.2.2","llguidance":"1.3.0","llvmlite":"0.44.0","lm-format-enforcer":"0.11.3","lmcache":"0.4.0","loguru":"0.7.3","markdown-it-py":"4.0.0","markupsafe":"3.0.3","mcp":"1.26.0","mdurl":"0.1.2","mistral-common":"1.9.1","modal":"1.5.4","model-hosting-container-standards":"0.1.13","modelscope":"1.34.0","mooncake-transfer-engine":"0.3.9","mpmath":"1.3.0","msgpack":"1.1.2","msgspec":"0.20.0","multidict":"6.7.1","networkx":"3.6.1","ninja":"1.13.0","nixl":"0.9.0","nixl-cu12":"0.10.1","numba":"0.61.2","numpy":"2.2.6","nvidia-cublas-cu12":"12.9.1.4","nvidia-cuda-cupti-cu12":"12.9.79","nvidia-cuda-nvrtc-cu12":"12.9.86","nvidia-cuda-runtime-cu12":"12.9.79","nvidia-cudnn-cu12":"9.10.2.21","nvidia-cudnn-frontend":"1.18.0","nvidia-cufft-cu12":"11.4.1.4","nvidia-cufile-cu12":"1.14.1.1","nvidia-curand-cu12":"10.3.10.19","nvidia-cusolver-cu12":"11.7.5.82","nvidia-cusparse-cu12":"12.5.10.65","nvidia-cusparselt-cu12":"0.7.1","nvidia-cutlass-dsl":"4.4.1","nvidia-cutlass-dsl-libs-base":"4.4.1","nvidia-ml-py":"13.590.48","nvidia-nccl-cu12":"2.27.5","nvidia-nvjitlink-cu12":"12.9.86","nvidia-nvshmem-cu12":"3.4.5","nvidia-nvtx-cu12":"12.9.79","nvtx":"0.2.14","openai":"2.24.0","openai-harmony":"0.0.8","opencv-python-headless":"4.13.0.92","opentelemetry-api":"1.40.0","opentelemetry-exporter-otlp":"1.40.0","opentelemetry-exporter-otlp-proto-common":"1.40.0","opentelemetry-exporter-otlp-proto-grpc":"1.40.0","opentelemetry-exporter-otlp-proto-http":"1.40.0","opentelemetry-proto":"1.40.0","opentelemetry-sdk":"1.40.0","opentelemetry-semantic-conventions":"0.61b0","opentelemetry-semantic-conventions-ai":"0.4.15","outlines-core":"0.2.11","packaging":"26.0","partial-json-parser":"0.2.1.1.post7","pillow":"12.1.1","pip":"26.0.1","prometheus-client":"0.24.1","prometheus-fastapi-instrumentator":"7.1.0","propcache":"0.4.1","proto-plus":"1.27.1","protobuf":"6.33.5","psutil":"7.2.2","py-cpuinfo":"9.0.0","pyasn1":"0.6.2","pyasn1-modules":"0.4.2","pybase64":"1.4.3","pycountry":"26.2.16","pycparser":"3.0","pydantic":"2.12.5","pydantic-core":"2.41.5","pydantic-extra-types":"2.11.0","pydantic-settings":"2.13.1","pygments":"2.19.2","pyjwt":"2.11.0","python-dateutil":"2.9.0.post0","python-dotenv":"1.2.2","python-json-logger":"4.0.0","python-multipart":"0.0.22","pyyaml":"6.0.3","pyzmq":"27.1.0","quack-kernels":"0.3.2","ray":"2.54.0","redis":"7.3.0","referencing":"0.37.0","regex":"2026.2.28","requests":"2.32.5","rich":"14.3.3","rich-toolkit":"0.19.7","rignore":"0.7.6","rpds-py":"0.30.0","rsa":"4.9.1","runai-model-streamer":"0.15.7","runai-model-streamer-gcs":"0.15.7","runai-model-streamer-s3":"0.15.7","s3transfer":"0.16.0","safetensors":"0.7.0","sentencepiece":"0.2.1","sentry-sdk":"2.54.0","setproctitle":"1.3.7","setuptools":"80.10.2","setuptools-scm":"9.2.2","shellingham":"1.5.4","six":"1.17.0","sniffio":"1.3.1","sortedcontainers":"2.4.0","sse-starlette":"3.3.2","starlette":"0.52.1","supervisor":"4.3.0","sympy":"1.14.0","synaptic-tuner":"1.1.0","synchronicity":"0.12.5","tabulate":"0.10.0","tiktoken":"0.12.0","timm":"1.0.25","tokenizers":"0.22.2","toml":"0.10.2","torch":"2.10.0+cu129","torch-c-dlpack-ext":"0.1.5","torchaudio":"2.10.0+cu129","torchvision":"0.25.0+cu129","tqdm":"4.67.3","transformers":"4.57.6","triton":"3.6.0","typer":"0.24.1","types-certifi":"2021.10.8.3","types-toml":"0.10.8.20260518","typing-extensions":"4.15.0","typing-inspection":"0.4.2","urllib3":"2.6.3","uv":"0.10.9","uvicorn":"0.41.0","uvloop":"0.22.1","vllm":"0.17.1","watchfiles":"1.1.1","websockets":"16.0","xgrammar":"0.1.29","yarl":"1.23.0","zipp":"3.23.0"},"python":{"executable":"/opt/synaptic-inference/bin/python","executable_sha256":"185be5c01624944b5c422a8d1e4995a7df010036a99c9f9d84a80a09ae1bd72d","implementation":"cpython","version":"3.12.13"},"schema_version":"synaptic-modal-inference-runtime-lock/v1","sdk_version":"1.5.4","source_inventory":[{"path":"Evaluator/__init__.py","sha256":"d1b5aea793a95c47618d4b2469a8bae0b14fd11f81501c1d71546469a39a5cbd","size_bytes":3061},{"path":"Evaluator/base_client.py","sha256":"ac7e77909bdf161a4fd9b1ad2e1e2cfe085ec31f36baa6b19f8976f7b2358cbd","size_bytes":19362},{"path":"Evaluator/config.py","sha256":"94fbe8e198d065141ff9ad002b0a6e26e56d865fc1e3a3fb23479a8e69d09303","size_bytes":21956},{"path":"Evaluator/enums.py","sha256":"2ced9426d860e9ec410e376d9cf853d75e8e111e78dd0bee31e25bee295e4854","size_bytes":3371},{"path":"Evaluator/openai_compat_client.py","sha256":"6ac6211185d2c57d3b31022d31478e3253046fd6cb4de1903d8d0b209bad3a79","size_bytes":5174},{"path":"Evaluator/protocols.py","sha256":"239fe049b5fe4c9e9861aede487feac1f06a7642891378a1ccfba444072ad706","size_bytes":4560},{"path":"Evaluator/verified_vllm_chat.py","sha256":"f130e47d661aa77d68c104084fafe9e5df3af88684ef5a3139f4d53ff2e32465","size_bytes":5916},{"path":"Evaluator/vllm_client.py","sha256":"4e88d2055dc1b32cc22a1393cdf038ccd76d91d915a8be0774b0a8ec87bd4d1e","size_bytes":1995},{"path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"path":"synaptic_tuner/api/v1/planning.py","sha256":"e715b105a513fa56cfa510416e725471f8f3100cb34257be74bef3e40ca78460","size_bytes":9569},{"path":"synaptic_tuner/api/v1/providers.py","sha256":"eca5efd5a6dee453543ae338ffddf9451d34ce1743e4ab4bde024b47be571ec0","size_bytes":4940},{"path":"synaptic_tuner/api/v1/results.py","sha256":"65e530325c92c7afa18e3ca5f2ad9a11baf7d590221f08c33badfbf2d5f2c7b2","size_bytes":2236},{"path":"synaptic_tuner/api/v1/runs_facade.py","sha256":"bf6155726c99cfc5574385009e20f4f65b69a9ad4b96e33cf7065263d1d4c455","size_bytes":23572},{"path":"synaptic_tuner/api/v1/training_facade.py","sha256":"bf094d0e152f5861bf8a8ff51962e0700123d9b43ed91d914b9e78bbea6facd9","size_bytes":11118},{"path":"synaptic_tuner/api/v1/training_input.py","sha256":"209326325f8796ab7d038878e92def2c5521d7d98b7b9cc4dd1766d2e4f68c9d","size_bytes":38650},{"path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"path":"tuner/cloud/runtime_layout.py","sha256":"2a4cd1f5fc63d95bb3f0b5bc4f44e616bb0308d9509aceb198943953746ff705","size_bytes":9435},{"path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"path":"tuner/execution/_effect_executor.py","sha256":"7aa3c523d971486417fe735c62a5cb3fe7e0698caa514d0da047293ad6b3e9cd","size_bytes":500},{"path":"tuner/execution/broker.py","sha256":"c504bd734b41ef9785fe5fa591966bb0b0bce3b43dcce0695b4ee0c142ca3d1e","size_bytes":6305},{"path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"path":"tuner/execution/coordinator_v1/__init__.py","sha256":"453718a1827cfb23e7ca0c908d3d256f62ca3e3fff8f75ed8a785f8336b17bbe","size_bytes":66},{"path":"tuner/execution/coordinator_v1/coordinator.py","sha256":"9bc8ecb60d472ad0b3c0d3192d8c73d88eb28dc43af55677fbdd4b3673b71ef9","size_bytes":47018},{"path":"tuner/execution/coordinator_v1/cursors.py","sha256":"5f7c899064dd26959aeed6b328b236b01f7ff549e2d29348bd45dc0480ede26b","size_bytes":9088},{"path":"tuner/execution/coordinator_v1/foundation.py","sha256":"4433c390075544df5516a5e213af4b4b697485ff7627f5a6dcace21a98029f94","size_bytes":12972},{"path":"tuner/execution/coordinator_v1/model.py","sha256":"1d2150d588f6b84f1b58913df398365a9491d6486a8a871bb2a22fa98cafc722","size_bytes":75837},{"path":"tuner/execution/coordinator_v1/ports.py","sha256":"7a028c32c3a3eab905bbc202248b4bf39411cd22e73fddacf5099b7d15dc3832","size_bytes":9343},{"path":"tuner/execution/coordinator_v1/state_machine.py","sha256":"e5e63db3a85fb9fc5b1d7bb5444691f88f308cfb0428842c0801b06d6f2fbb3a","size_bytes":50176},{"path":"tuner/execution/coordinator_v1/stores.py","sha256":"3a875f9e9d7d7d9899d5deba2f4d68cfada12bea499d5897592c9383ba2b16dc","size_bytes":39045},{"path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"path":"tuner/execution/foundation_v2/__init__.py","sha256":"1651bf897879603a83e7824f60d369d52ac5d0308c96f988c24465f33112cd77","size_bytes":77},{"path":"tuner/execution/foundation_v2/authority.py","sha256":"a0eb5b3537a59c7b4d33e9b8617abd072376545d808f1989f562704dfa1ad024","size_bytes":9958},{"path":"tuner/execution/foundation_v2/canonical.py","sha256":"80620a6088d137bfacb210e000c406632bf569de66350628fb0e0aee612babfb","size_bytes":4047},{"path":"tuner/execution/foundation_v2/commands.py","sha256":"0fff7f05401094a60a390514622bbaa5649a8fb1b5f85f938e856192dac34e5c","size_bytes":8891},{"path":"tuner/execution/foundation_v2/executors.py","sha256":"27a428368c7aa4b9b502ae8028de03b48b817ea9d237a08dd112e7a6afc20454","size_bytes":5331},{"path":"tuner/execution/foundation_v2/identities.py","sha256":"720ae3938627f075c181eb5a125d118df87c9d6f7f24869107c74df9fbc44388","size_bytes":2794},{"path":"tuner/execution/foundation_v2/observations.py","sha256":"3642e4fd35478be5cb9a41df8f758ec4581d9f341f470bad1a4a2fbf8093636c","size_bytes":2566},{"path":"tuner/execution/foundation_v2/operations.py","sha256":"0f3c12d916a71a1fe20017b5ff7e99676c0acbf8bae7fac903d629c1aa3a9e6d","size_bytes":2259},{"path":"tuner/execution/foundation_v2/preparation.py","sha256":"09883f2455e521a7baa9b30fbb81518aa33ab2845392009d87cb54606ff80f1d","size_bytes":3575},{"path":"tuner/execution/foundation_v2/receipts.py","sha256":"c327476257236209ea1e29258fa55d90859bb262d27c132870d21399dfb5f626","size_bytes":15339},{"path":"tuner/execution/foundation_v2/references.py","sha256":"53d8e2070395a4c7f5c5a9361532f7a78de714003c760b55a23366394561b37f","size_bytes":3336},{"path":"tuner/execution/foundation_v2/repository.py","sha256":"d311b21fe8f2ed4b85d50ffa55fca370fe6452841d7a0189c5d56b841f69ef7f","size_bytes":57985},{"path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"path":"tuner/execution/operation.py","sha256":"9f86c585fc7ac3f7b2689a4701d598e32a382c3d1e0b87a7732ef35235568504","size_bytes":13702},{"path":"tuner/execution/providers/__init__.py","sha256":"8848731f5d1877c77ea8fcb66a82b85196aeafc06ba0e7463eca291ec1a5f41d","size_bytes":252},{"path":"tuner/execution/providers/contracts.py","sha256":"a21f22c8d373037d6426cd94dda48d7252438a08e115120188980dcaa34d6361","size_bytes":11506},{"path":"tuner/execution/providers/modal/__init__.py","sha256":"0f4e16bed9b4d05c59a5eef1d0ddc061720d5817a68a2bbe1073b20a19c27319","size_bytes":104},{"path":"tuner/execution/providers/modal/binding.py","sha256":"2d2acb7685798f487988b7690c1dbaf6ddc49e34ed8cd3600add438e17fc9ad5","size_bytes":1810},{"path":"tuner/execution/providers/modal/config.py","sha256":"bfa32ed26f3b8ece5e89e1a3524cb81e85ae3240e72ce93430ffb82dbb02f525","size_bytes":20788},{"path":"tuner/execution/providers/modal/contracts.py","sha256":"249e75a9ad13b93c65ac95cb100dc4fe24ed4c59674770832af3d5a9c4112dd2","size_bytes":6449},{"path":"tuner/execution/providers/modal/control.py","sha256":"19e05126196be5009c2cedac36a39008e82ca0bd17c9ec461f68f6ef26ece750","size_bytes":10359},{"path":"tuner/execution/providers/modal/coordinator_adapter.py","sha256":"619ba5e4e10f1a26ad4c154490da9328f9863581df02bf768e48e045ba46e413","size_bytes":13178},{"path":"tuner/execution/providers/modal/coordinator_binding.py","sha256":"e9267c37dce7273ffcad5f1db5332e205ba5282a9fbad4a647581e23c08b821e","size_bytes":4125},{"path":"tuner/execution/providers/modal/coordinator_bundle.py","sha256":"5d8a10d2c4a0dce83174eeca202c377730820ced6945be6f7717e3a0cfd91eb0","size_bytes":31316},{"path":"tuner/execution/providers/modal/coordinator_dispatch.py","sha256":"8e5bc475c6b7d15345e22e1f8cebdabcffab5a01ffefb7d4acbfb073be02b49f","size_bytes":10571},{"path":"tuner/execution/providers/modal/coordinator_launch.py","sha256":"5975bee2d73dd09e02008ede368ebc8f03325815e8bcfd4c093736214d5f0956","size_bytes":10982},{"path":"tuner/execution/providers/modal/coordinator_preflight.py","sha256":"f452e906d37fa9dba932e9df46f829447c95ec276191e34ec3e46831c89da875","size_bytes":13746},{"path":"tuner/execution/providers/modal/coordinator_reader.py","sha256":"92a1c21725587a88435d30b6b687f63711a12ad4c77345f0776521d6503103f4","size_bytes":28524},{"path":"tuner/execution/providers/modal/coordinator_staging.py","sha256":"de47b688e859ab96a612c01b1c7869d814e40b83ae81dcd7f2b1132e18c794f0","size_bytes":20880},{"path":"tuner/execution/providers/modal/coordinator_submit_preparation.py","sha256":"28814779c7f086016ae3ff617e0f1d4cbd8b52f518d3d889a905ea03d70456ec","size_bytes":5213},{"path":"tuner/execution/providers/modal/coordinator_wire.py","sha256":"07d46b92b33e30cb45bb6b1139ce4f80afdb09e1a17a3f3c2526ddcfae6983c9","size_bytes":12998},{"path":"tuner/execution/providers/modal/deployment_identity.py","sha256":"1b7007e5a9c11abdfe0ad4f106f742faa0ee556921b4017d3b48cb772bd0da4f","size_bytes":1196},{"path":"tuner/execution/providers/modal/deployment_v1.py","sha256":"579ace8adbddfe30e09d5a39df9926a54b111a139c704fc7efc7cebf533acf94","size_bytes":3628},{"path":"tuner/execution/providers/modal/facade.py","sha256":"55b89b3d0faff6238eeead068d6963d26f27749d725d2f2b8ae32449b348e32d","size_bytes":17704},{"path":"tuner/execution/providers/modal/inference-dependencies.lock","sha256":"ca4e3f0d7fa06533ad1127d84b539279791e37f07dc62cc81fbf0edd3a59de88","size_bytes":8458},{"path":"tuner/execution/providers/modal/inference-worker-closure.json","sha256":"8bf246adc7d6a0b7e19d99f4747a660c4efd42d5c11e2f277ec2bcd7a47c5d26","size_bytes":18118},{"path":"tuner/execution/providers/modal/inference_artifacts.py","sha256":"5816f0c5662de8edad752554ba149d9527ced40ebdceeccc993c136ab099bff3","size_bytes":14973},{"path":"tuner/execution/providers/modal/inference_binding.py","sha256":"f2e8c4f1d8fc1f5e70702c88cbfff044a1f4f569e06b9a31c33a0cb9aaaf5894","size_bytes":23363},{"path":"tuner/execution/providers/modal/inference_bootstrap.py","sha256":"db2f09c1dfb2aee667b46fbb6566be8b2922d8e762896e5577d5c5078ad7d122","size_bytes":12546},{"path":"tuner/execution/providers/modal/inference_channel.py","sha256":"23492b4c7f48d86d23fb21ffd3902ac3f48bae381c30ea4759c803577911592e","size_bytes":13438},{"path":"tuner/execution/providers/modal/inference_commands.py","sha256":"1fa2dd667cb3b33e012d37b0688bfe08fc7a7cec52230a202f1f9ddda70110c5","size_bytes":4977},{"path":"tuner/execution/providers/modal/inference_entrypoint.py","sha256":"6ec8bdcbbf6d87fb5fc02a2c42d2c0f344ea965ccc4c0d94a630055c6e779991","size_bytes":10873},{"path":"tuner/execution/providers/modal/inference_model.py","sha256":"c08e1a35bb71a11c3466aedfbd8d3de87cf32d6cbd85820c3dcde2fefd14fd65","size_bytes":5935},{"path":"tuner/execution/providers/modal/inference_preparation.py","sha256":"43a1c883a8e5b2f25364e3a20f59d92a852af15fda0c9d506c213c0bf185fc32","size_bytes":49436},{"path":"tuner/execution/providers/modal/inference_runtime.py","sha256":"f8ba0d5744f90ca95e1fea3e5faf293e153a91047ffb09596f0305e1339d56ee","size_bytes":19283},{"path":"tuner/execution/providers/modal/inference_wire.py","sha256":"0492c10c6a384596bba04b3b409dc34f1f1a4afa88b7321d0d1a6fac5e29a1fb","size_bytes":20150},{"path":"tuner/execution/providers/modal/inference_worker.py","sha256":"cfb27e938c60757ef08c80b918fb522012c84918e7a0cd41a63de2cb2486cb87","size_bytes":15657},{"path":"tuner/execution/providers/modal/inference_workload.py","sha256":"4eaf399485e76829a98f57907c012658ce41d02745978827f21262ce2be94ff8","size_bytes":20421},{"path":"tuner/execution/providers/modal/manifest.py","sha256":"fc2ab4dcc46d136a681bf124d7a9fc3e5deedeeb3641d798fd207cdb62bb506f","size_bytes":8579},{"path":"tuner/execution/providers/modal/model_snapshot.py","sha256":"0b6d665c347a65150bff21b8b21c19c1f437bbcd8db69821c4240f2c57e44cdf","size_bytes":14751},{"path":"tuner/execution/providers/modal/mounted_io.py","sha256":"e49da2e749d3be1ca19fdc7f581b29fb6d1e748a86fffcf0fde2be5cdc1a0d5b","size_bytes":13076},{"path":"tuner/execution/providers/modal/prepared_input.py","sha256":"c16cef5cfcc73ab9a7826977b26e18912ac46409d91b9eae165893ed265896ac","size_bytes":8597},{"path":"tuner/execution/providers/modal/resolution.py","sha256":"5c7268aa644103ae0712b03ed55918527d6e2ed6f4120ef169866b0ffd1cbf6f","size_bytes":30179},{"path":"tuner/execution/providers/modal/runtime.py","sha256":"ada13733021a6dab19331da80076df630055abc5bddc192a34a5cda65ac718c4","size_bytes":14810},{"path":"tuner/execution/providers/modal/worker_ports.py","sha256":"adf334368dbb75e40177078cd32aedc445cd9dde8b5ece2401319d472dbccdb0","size_bytes":2928},{"path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"path":"tuner/inference/__init__.py","sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","size_bytes":0},{"path":"tuner/inference/chat_session.py","sha256":"b95a829f834406a0a0c152f9af713225fe205a9225ad7b0f959b1ba801dbcac2","size_bytes":22355},{"path":"tuner/inference/owned_process.py","sha256":"f4cbb4e1c2305615b2c127e1103f60b1e17e955fe20a2a847b1d88b7ba8c4c98","size_bytes":8832},{"path":"tuner/inference/retrieved_model.py","sha256":"4a8cf2b2cbe4f03bf7c4d240c5e59a6ec6d20826c686b3a06c0bc1642fa234d8","size_bytes":31703},{"path":"tuner/inference/run_chat.py","sha256":"9263d325224bee4b9a0e7c16045e30766d9a2e2d2b550f0508040161687d3cfc","size_bytes":6228},{"path":"tuner/inference/serving_target.py","sha256":"50ec8054d1d6d08724d7a29183ca54a04d9c915d2c04274bdac6834764bdb800","size_bytes":12227},{"path":"tuner/inference/vllm_runtime.py","sha256":"fafbcfdeee78143a2760e5fbf29621897e9be014814eefb69f1167b188c296c1","size_bytes":26019},{"path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"path":"tuner/runtime/dispatch.py","sha256":"4c41faf87d7e620bdc281490bf3725550b83dacca8b15ea47bab253f669cdd8e","size_bytes":35463},{"path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"path":"tuner/runtime/verification.py","sha256":"4c2314b20cbf7937a3f24aa90676000b9e21f0ab6701b2c1225a53e5596bf4a0","size_bytes":53856},{"path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"path":"tuner/training/contracts.py","sha256":"91c4d66d9810cc0471981c3d4ef4b54feaed5b92eb41a468f31201f8111303b5","size_bytes":21504},{"path":"tuner/training/coordinator_material.py","sha256":"c563aa3b1fbd940ddd3b8e56e4f33de70675cb831e938175f40debc0645fd9c7","size_bytes":12950},{"path":"tuner/training/grpo_compilation.py","sha256":"c6268486375bbc16f1888bb43d07b93a8de280d000a774914d59d7fd05ca5cc0","size_bytes":8998},{"path":"tuner/training/methods/__init__.py","sha256":"99db689246273707ae3df5087859ebb510e84b664f9f173c0d5827abc3e6a3ea","size_bytes":361},{"path":"tuner/training/methods/sft.py","sha256":"b501f9c214dec4b0b1550543281c25aba8dbb102604df253c641c85f7cae587b","size_bytes":9997},{"path":"tuner/training/packaged_compilation.py","sha256":"477caede6902fc587af680889b93fd9e0e42493d1a6653f3cd7413a9c7cadde8","size_bytes":9413},{"path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722},{"path":"tuner/training/post_training.py","sha256":"ba8c2ea3fe49a647d502c6a874a44658cc2e575d4cd5729966102b0ebb973a88","size_bytes":7971},{"path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"worker_closure_manifest_path":"tuner/execution/providers/modal/inference-worker-closure.json"} diff --git a/tuner/execution/providers/modal/inference-worker-closure.json b/tuner/execution/providers/modal/inference-worker-closure.json index 65c5551a..382c69a3 100644 --- a/tuner/execution/providers/modal/inference-worker-closure.json +++ b/tuner/execution/providers/modal/inference-worker-closure.json @@ -1 +1 @@ -{"closure_digest":"11b8b9df877b6dbb3a50268f7782e8ed0f6b291e85a5b4981ff0a2f8edf47175","entrypoint":"tuner/execution/providers/modal/inference_bootstrap.py","member_count":124,"members":[{"path":"Evaluator/__init__.py","sha256":"d1b5aea793a95c47618d4b2469a8bae0b14fd11f81501c1d71546469a39a5cbd","size_bytes":3061},{"path":"Evaluator/base_client.py","sha256":"ac7e77909bdf161a4fd9b1ad2e1e2cfe085ec31f36baa6b19f8976f7b2358cbd","size_bytes":19362},{"path":"Evaluator/config.py","sha256":"94fbe8e198d065141ff9ad002b0a6e26e56d865fc1e3a3fb23479a8e69d09303","size_bytes":21956},{"path":"Evaluator/enums.py","sha256":"2ced9426d860e9ec410e376d9cf853d75e8e111e78dd0bee31e25bee295e4854","size_bytes":3371},{"path":"Evaluator/openai_compat_client.py","sha256":"6ac6211185d2c57d3b31022d31478e3253046fd6cb4de1903d8d0b209bad3a79","size_bytes":5174},{"path":"Evaluator/protocols.py","sha256":"239fe049b5fe4c9e9861aede487feac1f06a7642891378a1ccfba444072ad706","size_bytes":4560},{"path":"Evaluator/verified_vllm_chat.py","sha256":"f130e47d661aa77d68c104084fafe9e5df3af88684ef5a3139f4d53ff2e32465","size_bytes":5916},{"path":"Evaluator/vllm_client.py","sha256":"4e88d2055dc1b32cc22a1393cdf038ccd76d91d915a8be0774b0a8ec87bd4d1e","size_bytes":1995},{"path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"path":"synaptic_tuner/api/v1/planning.py","sha256":"e715b105a513fa56cfa510416e725471f8f3100cb34257be74bef3e40ca78460","size_bytes":9569},{"path":"synaptic_tuner/api/v1/providers.py","sha256":"eca5efd5a6dee453543ae338ffddf9451d34ce1743e4ab4bde024b47be571ec0","size_bytes":4940},{"path":"synaptic_tuner/api/v1/results.py","sha256":"65e530325c92c7afa18e3ca5f2ad9a11baf7d590221f08c33badfbf2d5f2c7b2","size_bytes":2236},{"path":"synaptic_tuner/api/v1/runs_facade.py","sha256":"bf6155726c99cfc5574385009e20f4f65b69a9ad4b96e33cf7065263d1d4c455","size_bytes":23572},{"path":"synaptic_tuner/api/v1/training_facade.py","sha256":"bf094d0e152f5861bf8a8ff51962e0700123d9b43ed91d914b9e78bbea6facd9","size_bytes":11118},{"path":"synaptic_tuner/api/v1/training_input.py","sha256":"209326325f8796ab7d038878e92def2c5521d7d98b7b9cc4dd1766d2e4f68c9d","size_bytes":38650},{"path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"path":"tuner/cloud/runtime_layout.py","sha256":"2a4cd1f5fc63d95bb3f0b5bc4f44e616bb0308d9509aceb198943953746ff705","size_bytes":9435},{"path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"path":"tuner/execution/_effect_executor.py","sha256":"7aa3c523d971486417fe735c62a5cb3fe7e0698caa514d0da047293ad6b3e9cd","size_bytes":500},{"path":"tuner/execution/broker.py","sha256":"c504bd734b41ef9785fe5fa591966bb0b0bce3b43dcce0695b4ee0c142ca3d1e","size_bytes":6305},{"path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"path":"tuner/execution/coordinator_v1/__init__.py","sha256":"453718a1827cfb23e7ca0c908d3d256f62ca3e3fff8f75ed8a785f8336b17bbe","size_bytes":66},{"path":"tuner/execution/coordinator_v1/coordinator.py","sha256":"9bc8ecb60d472ad0b3c0d3192d8c73d88eb28dc43af55677fbdd4b3673b71ef9","size_bytes":47018},{"path":"tuner/execution/coordinator_v1/cursors.py","sha256":"5f7c899064dd26959aeed6b328b236b01f7ff549e2d29348bd45dc0480ede26b","size_bytes":9088},{"path":"tuner/execution/coordinator_v1/foundation.py","sha256":"4433c390075544df5516a5e213af4b4b697485ff7627f5a6dcace21a98029f94","size_bytes":12972},{"path":"tuner/execution/coordinator_v1/model.py","sha256":"1d2150d588f6b84f1b58913df398365a9491d6486a8a871bb2a22fa98cafc722","size_bytes":75837},{"path":"tuner/execution/coordinator_v1/ports.py","sha256":"7a028c32c3a3eab905bbc202248b4bf39411cd22e73fddacf5099b7d15dc3832","size_bytes":9343},{"path":"tuner/execution/coordinator_v1/state_machine.py","sha256":"e5e63db3a85fb9fc5b1d7bb5444691f88f308cfb0428842c0801b06d6f2fbb3a","size_bytes":50176},{"path":"tuner/execution/coordinator_v1/stores.py","sha256":"3a875f9e9d7d7d9899d5deba2f4d68cfada12bea499d5897592c9383ba2b16dc","size_bytes":39045},{"path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"path":"tuner/execution/foundation_v2/__init__.py","sha256":"1651bf897879603a83e7824f60d369d52ac5d0308c96f988c24465f33112cd77","size_bytes":77},{"path":"tuner/execution/foundation_v2/authority.py","sha256":"a0eb5b3537a59c7b4d33e9b8617abd072376545d808f1989f562704dfa1ad024","size_bytes":9958},{"path":"tuner/execution/foundation_v2/canonical.py","sha256":"80620a6088d137bfacb210e000c406632bf569de66350628fb0e0aee612babfb","size_bytes":4047},{"path":"tuner/execution/foundation_v2/commands.py","sha256":"0fff7f05401094a60a390514622bbaa5649a8fb1b5f85f938e856192dac34e5c","size_bytes":8891},{"path":"tuner/execution/foundation_v2/executors.py","sha256":"27a428368c7aa4b9b502ae8028de03b48b817ea9d237a08dd112e7a6afc20454","size_bytes":5331},{"path":"tuner/execution/foundation_v2/identities.py","sha256":"720ae3938627f075c181eb5a125d118df87c9d6f7f24869107c74df9fbc44388","size_bytes":2794},{"path":"tuner/execution/foundation_v2/observations.py","sha256":"3642e4fd35478be5cb9a41df8f758ec4581d9f341f470bad1a4a2fbf8093636c","size_bytes":2566},{"path":"tuner/execution/foundation_v2/operations.py","sha256":"0f3c12d916a71a1fe20017b5ff7e99676c0acbf8bae7fac903d629c1aa3a9e6d","size_bytes":2259},{"path":"tuner/execution/foundation_v2/preparation.py","sha256":"09883f2455e521a7baa9b30fbb81518aa33ab2845392009d87cb54606ff80f1d","size_bytes":3575},{"path":"tuner/execution/foundation_v2/receipts.py","sha256":"c327476257236209ea1e29258fa55d90859bb262d27c132870d21399dfb5f626","size_bytes":15339},{"path":"tuner/execution/foundation_v2/references.py","sha256":"53d8e2070395a4c7f5c5a9361532f7a78de714003c760b55a23366394561b37f","size_bytes":3336},{"path":"tuner/execution/foundation_v2/repository.py","sha256":"d311b21fe8f2ed4b85d50ffa55fca370fe6452841d7a0189c5d56b841f69ef7f","size_bytes":57985},{"path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"path":"tuner/execution/operation.py","sha256":"9f86c585fc7ac3f7b2689a4701d598e32a382c3d1e0b87a7732ef35235568504","size_bytes":13702},{"path":"tuner/execution/providers/__init__.py","sha256":"8848731f5d1877c77ea8fcb66a82b85196aeafc06ba0e7463eca291ec1a5f41d","size_bytes":252},{"path":"tuner/execution/providers/contracts.py","sha256":"a21f22c8d373037d6426cd94dda48d7252438a08e115120188980dcaa34d6361","size_bytes":11506},{"path":"tuner/execution/providers/modal/__init__.py","sha256":"0f4e16bed9b4d05c59a5eef1d0ddc061720d5817a68a2bbe1073b20a19c27319","size_bytes":104},{"path":"tuner/execution/providers/modal/binding.py","sha256":"2d2acb7685798f487988b7690c1dbaf6ddc49e34ed8cd3600add438e17fc9ad5","size_bytes":1810},{"path":"tuner/execution/providers/modal/config.py","sha256":"bfa32ed26f3b8ece5e89e1a3524cb81e85ae3240e72ce93430ffb82dbb02f525","size_bytes":20788},{"path":"tuner/execution/providers/modal/contracts.py","sha256":"249e75a9ad13b93c65ac95cb100dc4fe24ed4c59674770832af3d5a9c4112dd2","size_bytes":6449},{"path":"tuner/execution/providers/modal/control.py","sha256":"19e05126196be5009c2cedac36a39008e82ca0bd17c9ec461f68f6ef26ece750","size_bytes":10359},{"path":"tuner/execution/providers/modal/coordinator_adapter.py","sha256":"619ba5e4e10f1a26ad4c154490da9328f9863581df02bf768e48e045ba46e413","size_bytes":13178},{"path":"tuner/execution/providers/modal/coordinator_binding.py","sha256":"e9267c37dce7273ffcad5f1db5332e205ba5282a9fbad4a647581e23c08b821e","size_bytes":4125},{"path":"tuner/execution/providers/modal/coordinator_bundle.py","sha256":"5d8a10d2c4a0dce83174eeca202c377730820ced6945be6f7717e3a0cfd91eb0","size_bytes":31316},{"path":"tuner/execution/providers/modal/coordinator_dispatch.py","sha256":"8e5bc475c6b7d15345e22e1f8cebdabcffab5a01ffefb7d4acbfb073be02b49f","size_bytes":10571},{"path":"tuner/execution/providers/modal/coordinator_launch.py","sha256":"5975bee2d73dd09e02008ede368ebc8f03325815e8bcfd4c093736214d5f0956","size_bytes":10982},{"path":"tuner/execution/providers/modal/coordinator_preflight.py","sha256":"f452e906d37fa9dba932e9df46f829447c95ec276191e34ec3e46831c89da875","size_bytes":13746},{"path":"tuner/execution/providers/modal/coordinator_reader.py","sha256":"92a1c21725587a88435d30b6b687f63711a12ad4c77345f0776521d6503103f4","size_bytes":28524},{"path":"tuner/execution/providers/modal/coordinator_staging.py","sha256":"de47b688e859ab96a612c01b1c7869d814e40b83ae81dcd7f2b1132e18c794f0","size_bytes":20880},{"path":"tuner/execution/providers/modal/coordinator_submit_preparation.py","sha256":"28814779c7f086016ae3ff617e0f1d4cbd8b52f518d3d889a905ea03d70456ec","size_bytes":5213},{"path":"tuner/execution/providers/modal/coordinator_wire.py","sha256":"07d46b92b33e30cb45bb6b1139ce4f80afdb09e1a17a3f3c2526ddcfae6983c9","size_bytes":12998},{"path":"tuner/execution/providers/modal/deployment_identity.py","sha256":"1b7007e5a9c11abdfe0ad4f106f742faa0ee556921b4017d3b48cb772bd0da4f","size_bytes":1196},{"path":"tuner/execution/providers/modal/deployment_v1.py","sha256":"579ace8adbddfe30e09d5a39df9926a54b111a139c704fc7efc7cebf533acf94","size_bytes":3628},{"path":"tuner/execution/providers/modal/facade.py","sha256":"55b89b3d0faff6238eeead068d6963d26f27749d725d2f2b8ae32449b348e32d","size_bytes":17704},{"path":"tuner/execution/providers/modal/inference_artifacts.py","sha256":"5816f0c5662de8edad752554ba149d9527ced40ebdceeccc993c136ab099bff3","size_bytes":14973},{"path":"tuner/execution/providers/modal/inference_binding.py","sha256":"f2e8c4f1d8fc1f5e70702c88cbfff044a1f4f569e06b9a31c33a0cb9aaaf5894","size_bytes":23363},{"path":"tuner/execution/providers/modal/inference_bootstrap.py","sha256":"db2f09c1dfb2aee667b46fbb6566be8b2922d8e762896e5577d5c5078ad7d122","size_bytes":12546},{"path":"tuner/execution/providers/modal/inference_channel.py","sha256":"23492b4c7f48d86d23fb21ffd3902ac3f48bae381c30ea4759c803577911592e","size_bytes":13438},{"path":"tuner/execution/providers/modal/inference_commands.py","sha256":"1fa2dd667cb3b33e012d37b0688bfe08fc7a7cec52230a202f1f9ddda70110c5","size_bytes":4977},{"path":"tuner/execution/providers/modal/inference_entrypoint.py","sha256":"6ec8bdcbbf6d87fb5fc02a2c42d2c0f344ea965ccc4c0d94a630055c6e779991","size_bytes":10873},{"path":"tuner/execution/providers/modal/inference_model.py","sha256":"c08e1a35bb71a11c3466aedfbd8d3de87cf32d6cbd85820c3dcde2fefd14fd65","size_bytes":5935},{"path":"tuner/execution/providers/modal/inference_preparation.py","sha256":"43a1c883a8e5b2f25364e3a20f59d92a852af15fda0c9d506c213c0bf185fc32","size_bytes":49436},{"path":"tuner/execution/providers/modal/inference_runtime.py","sha256":"f8ba0d5744f90ca95e1fea3e5faf293e153a91047ffb09596f0305e1339d56ee","size_bytes":19283},{"path":"tuner/execution/providers/modal/inference_wire.py","sha256":"0492c10c6a384596bba04b3b409dc34f1f1a4afa88b7321d0d1a6fac5e29a1fb","size_bytes":20150},{"path":"tuner/execution/providers/modal/inference_worker.py","sha256":"cfb27e938c60757ef08c80b918fb522012c84918e7a0cd41a63de2cb2486cb87","size_bytes":15657},{"path":"tuner/execution/providers/modal/inference_workload.py","sha256":"4eaf399485e76829a98f57907c012658ce41d02745978827f21262ce2be94ff8","size_bytes":20421},{"path":"tuner/execution/providers/modal/manifest.py","sha256":"fc2ab4dcc46d136a681bf124d7a9fc3e5deedeeb3641d798fd207cdb62bb506f","size_bytes":8579},{"path":"tuner/execution/providers/modal/model_snapshot.py","sha256":"9d4762b821b2944a8a62932d0aec105c2f6920d64bf4ceed6907e5458799e337","size_bytes":14166},{"path":"tuner/execution/providers/modal/mounted_io.py","sha256":"e49da2e749d3be1ca19fdc7f581b29fb6d1e748a86fffcf0fde2be5cdc1a0d5b","size_bytes":13076},{"path":"tuner/execution/providers/modal/prepared_input.py","sha256":"c16cef5cfcc73ab9a7826977b26e18912ac46409d91b9eae165893ed265896ac","size_bytes":8597},{"path":"tuner/execution/providers/modal/resolution.py","sha256":"5c7268aa644103ae0712b03ed55918527d6e2ed6f4120ef169866b0ffd1cbf6f","size_bytes":30179},{"path":"tuner/execution/providers/modal/runtime.py","sha256":"ada13733021a6dab19331da80076df630055abc5bddc192a34a5cda65ac718c4","size_bytes":14810},{"path":"tuner/execution/providers/modal/worker_ports.py","sha256":"adf334368dbb75e40177078cd32aedc445cd9dde8b5ece2401319d472dbccdb0","size_bytes":2928},{"path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"path":"tuner/inference/__init__.py","sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","size_bytes":0},{"path":"tuner/inference/chat_session.py","sha256":"b95a829f834406a0a0c152f9af713225fe205a9225ad7b0f959b1ba801dbcac2","size_bytes":22355},{"path":"tuner/inference/owned_process.py","sha256":"f4cbb4e1c2305615b2c127e1103f60b1e17e955fe20a2a847b1d88b7ba8c4c98","size_bytes":8832},{"path":"tuner/inference/retrieved_model.py","sha256":"4a8cf2b2cbe4f03bf7c4d240c5e59a6ec6d20826c686b3a06c0bc1642fa234d8","size_bytes":31703},{"path":"tuner/inference/run_chat.py","sha256":"9263d325224bee4b9a0e7c16045e30766d9a2e2d2b550f0508040161687d3cfc","size_bytes":6228},{"path":"tuner/inference/serving_target.py","sha256":"50ec8054d1d6d08724d7a29183ca54a04d9c915d2c04274bdac6834764bdb800","size_bytes":12227},{"path":"tuner/inference/vllm_runtime.py","sha256":"fafbcfdeee78143a2760e5fbf29621897e9be014814eefb69f1167b188c296c1","size_bytes":26019},{"path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"path":"tuner/runtime/dispatch.py","sha256":"4c41faf87d7e620bdc281490bf3725550b83dacca8b15ea47bab253f669cdd8e","size_bytes":35463},{"path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"path":"tuner/runtime/verification.py","sha256":"4c2314b20cbf7937a3f24aa90676000b9e21f0ab6701b2c1225a53e5596bf4a0","size_bytes":53856},{"path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"path":"tuner/training/contracts.py","sha256":"91c4d66d9810cc0471981c3d4ef4b54feaed5b92eb41a468f31201f8111303b5","size_bytes":21504},{"path":"tuner/training/coordinator_material.py","sha256":"c563aa3b1fbd940ddd3b8e56e4f33de70675cb831e938175f40debc0645fd9c7","size_bytes":12950},{"path":"tuner/training/grpo_compilation.py","sha256":"c6268486375bbc16f1888bb43d07b93a8de280d000a774914d59d7fd05ca5cc0","size_bytes":8998},{"path":"tuner/training/methods/__init__.py","sha256":"99db689246273707ae3df5087859ebb510e84b664f9f173c0d5827abc3e6a3ea","size_bytes":361},{"path":"tuner/training/methods/sft.py","sha256":"b501f9c214dec4b0b1550543281c25aba8dbb102604df253c641c85f7cae587b","size_bytes":9997},{"path":"tuner/training/packaged_compilation.py","sha256":"477caede6902fc587af680889b93fd9e0e42493d1a6653f3cd7413a9c7cadde8","size_bytes":9413},{"path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722},{"path":"tuner/training/post_training.py","sha256":"ba8c2ea3fe49a647d502c6a874a44658cc2e575d4cd5729966102b0ebb973a88","size_bytes":7971},{"path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"payload_bytes":1656803,"schema_version":"synaptic-modal-inference-worker-closure/v1"} +{"closure_digest":"3a860efff7074722299b7f9678ba60287ad8319cd6c82a1d991bdeaa54e07a89","entrypoint":"tuner/execution/providers/modal/inference_bootstrap.py","member_count":124,"members":[{"path":"Evaluator/__init__.py","sha256":"d1b5aea793a95c47618d4b2469a8bae0b14fd11f81501c1d71546469a39a5cbd","size_bytes":3061},{"path":"Evaluator/base_client.py","sha256":"ac7e77909bdf161a4fd9b1ad2e1e2cfe085ec31f36baa6b19f8976f7b2358cbd","size_bytes":19362},{"path":"Evaluator/config.py","sha256":"94fbe8e198d065141ff9ad002b0a6e26e56d865fc1e3a3fb23479a8e69d09303","size_bytes":21956},{"path":"Evaluator/enums.py","sha256":"2ced9426d860e9ec410e376d9cf853d75e8e111e78dd0bee31e25bee295e4854","size_bytes":3371},{"path":"Evaluator/openai_compat_client.py","sha256":"6ac6211185d2c57d3b31022d31478e3253046fd6cb4de1903d8d0b209bad3a79","size_bytes":5174},{"path":"Evaluator/protocols.py","sha256":"239fe049b5fe4c9e9861aede487feac1f06a7642891378a1ccfba444072ad706","size_bytes":4560},{"path":"Evaluator/verified_vllm_chat.py","sha256":"f130e47d661aa77d68c104084fafe9e5df3af88684ef5a3139f4d53ff2e32465","size_bytes":5916},{"path":"Evaluator/vllm_client.py","sha256":"4e88d2055dc1b32cc22a1393cdf038ccd76d91d915a8be0774b0a8ec87bd4d1e","size_bytes":1995},{"path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"path":"synaptic_tuner/api/v1/planning.py","sha256":"e715b105a513fa56cfa510416e725471f8f3100cb34257be74bef3e40ca78460","size_bytes":9569},{"path":"synaptic_tuner/api/v1/providers.py","sha256":"eca5efd5a6dee453543ae338ffddf9451d34ce1743e4ab4bde024b47be571ec0","size_bytes":4940},{"path":"synaptic_tuner/api/v1/results.py","sha256":"65e530325c92c7afa18e3ca5f2ad9a11baf7d590221f08c33badfbf2d5f2c7b2","size_bytes":2236},{"path":"synaptic_tuner/api/v1/runs_facade.py","sha256":"bf6155726c99cfc5574385009e20f4f65b69a9ad4b96e33cf7065263d1d4c455","size_bytes":23572},{"path":"synaptic_tuner/api/v1/training_facade.py","sha256":"bf094d0e152f5861bf8a8ff51962e0700123d9b43ed91d914b9e78bbea6facd9","size_bytes":11118},{"path":"synaptic_tuner/api/v1/training_input.py","sha256":"209326325f8796ab7d038878e92def2c5521d7d98b7b9cc4dd1766d2e4f68c9d","size_bytes":38650},{"path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"path":"tuner/cloud/runtime_layout.py","sha256":"2a4cd1f5fc63d95bb3f0b5bc4f44e616bb0308d9509aceb198943953746ff705","size_bytes":9435},{"path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"path":"tuner/execution/_effect_executor.py","sha256":"7aa3c523d971486417fe735c62a5cb3fe7e0698caa514d0da047293ad6b3e9cd","size_bytes":500},{"path":"tuner/execution/broker.py","sha256":"c504bd734b41ef9785fe5fa591966bb0b0bce3b43dcce0695b4ee0c142ca3d1e","size_bytes":6305},{"path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"path":"tuner/execution/coordinator_v1/__init__.py","sha256":"453718a1827cfb23e7ca0c908d3d256f62ca3e3fff8f75ed8a785f8336b17bbe","size_bytes":66},{"path":"tuner/execution/coordinator_v1/coordinator.py","sha256":"9bc8ecb60d472ad0b3c0d3192d8c73d88eb28dc43af55677fbdd4b3673b71ef9","size_bytes":47018},{"path":"tuner/execution/coordinator_v1/cursors.py","sha256":"5f7c899064dd26959aeed6b328b236b01f7ff549e2d29348bd45dc0480ede26b","size_bytes":9088},{"path":"tuner/execution/coordinator_v1/foundation.py","sha256":"4433c390075544df5516a5e213af4b4b697485ff7627f5a6dcace21a98029f94","size_bytes":12972},{"path":"tuner/execution/coordinator_v1/model.py","sha256":"1d2150d588f6b84f1b58913df398365a9491d6486a8a871bb2a22fa98cafc722","size_bytes":75837},{"path":"tuner/execution/coordinator_v1/ports.py","sha256":"7a028c32c3a3eab905bbc202248b4bf39411cd22e73fddacf5099b7d15dc3832","size_bytes":9343},{"path":"tuner/execution/coordinator_v1/state_machine.py","sha256":"e5e63db3a85fb9fc5b1d7bb5444691f88f308cfb0428842c0801b06d6f2fbb3a","size_bytes":50176},{"path":"tuner/execution/coordinator_v1/stores.py","sha256":"3a875f9e9d7d7d9899d5deba2f4d68cfada12bea499d5897592c9383ba2b16dc","size_bytes":39045},{"path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"path":"tuner/execution/foundation_v2/__init__.py","sha256":"1651bf897879603a83e7824f60d369d52ac5d0308c96f988c24465f33112cd77","size_bytes":77},{"path":"tuner/execution/foundation_v2/authority.py","sha256":"a0eb5b3537a59c7b4d33e9b8617abd072376545d808f1989f562704dfa1ad024","size_bytes":9958},{"path":"tuner/execution/foundation_v2/canonical.py","sha256":"80620a6088d137bfacb210e000c406632bf569de66350628fb0e0aee612babfb","size_bytes":4047},{"path":"tuner/execution/foundation_v2/commands.py","sha256":"0fff7f05401094a60a390514622bbaa5649a8fb1b5f85f938e856192dac34e5c","size_bytes":8891},{"path":"tuner/execution/foundation_v2/executors.py","sha256":"27a428368c7aa4b9b502ae8028de03b48b817ea9d237a08dd112e7a6afc20454","size_bytes":5331},{"path":"tuner/execution/foundation_v2/identities.py","sha256":"720ae3938627f075c181eb5a125d118df87c9d6f7f24869107c74df9fbc44388","size_bytes":2794},{"path":"tuner/execution/foundation_v2/observations.py","sha256":"3642e4fd35478be5cb9a41df8f758ec4581d9f341f470bad1a4a2fbf8093636c","size_bytes":2566},{"path":"tuner/execution/foundation_v2/operations.py","sha256":"0f3c12d916a71a1fe20017b5ff7e99676c0acbf8bae7fac903d629c1aa3a9e6d","size_bytes":2259},{"path":"tuner/execution/foundation_v2/preparation.py","sha256":"09883f2455e521a7baa9b30fbb81518aa33ab2845392009d87cb54606ff80f1d","size_bytes":3575},{"path":"tuner/execution/foundation_v2/receipts.py","sha256":"c327476257236209ea1e29258fa55d90859bb262d27c132870d21399dfb5f626","size_bytes":15339},{"path":"tuner/execution/foundation_v2/references.py","sha256":"53d8e2070395a4c7f5c5a9361532f7a78de714003c760b55a23366394561b37f","size_bytes":3336},{"path":"tuner/execution/foundation_v2/repository.py","sha256":"d311b21fe8f2ed4b85d50ffa55fca370fe6452841d7a0189c5d56b841f69ef7f","size_bytes":57985},{"path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"path":"tuner/execution/operation.py","sha256":"9f86c585fc7ac3f7b2689a4701d598e32a382c3d1e0b87a7732ef35235568504","size_bytes":13702},{"path":"tuner/execution/providers/__init__.py","sha256":"8848731f5d1877c77ea8fcb66a82b85196aeafc06ba0e7463eca291ec1a5f41d","size_bytes":252},{"path":"tuner/execution/providers/contracts.py","sha256":"a21f22c8d373037d6426cd94dda48d7252438a08e115120188980dcaa34d6361","size_bytes":11506},{"path":"tuner/execution/providers/modal/__init__.py","sha256":"0f4e16bed9b4d05c59a5eef1d0ddc061720d5817a68a2bbe1073b20a19c27319","size_bytes":104},{"path":"tuner/execution/providers/modal/binding.py","sha256":"2d2acb7685798f487988b7690c1dbaf6ddc49e34ed8cd3600add438e17fc9ad5","size_bytes":1810},{"path":"tuner/execution/providers/modal/config.py","sha256":"bfa32ed26f3b8ece5e89e1a3524cb81e85ae3240e72ce93430ffb82dbb02f525","size_bytes":20788},{"path":"tuner/execution/providers/modal/contracts.py","sha256":"249e75a9ad13b93c65ac95cb100dc4fe24ed4c59674770832af3d5a9c4112dd2","size_bytes":6449},{"path":"tuner/execution/providers/modal/control.py","sha256":"19e05126196be5009c2cedac36a39008e82ca0bd17c9ec461f68f6ef26ece750","size_bytes":10359},{"path":"tuner/execution/providers/modal/coordinator_adapter.py","sha256":"619ba5e4e10f1a26ad4c154490da9328f9863581df02bf768e48e045ba46e413","size_bytes":13178},{"path":"tuner/execution/providers/modal/coordinator_binding.py","sha256":"e9267c37dce7273ffcad5f1db5332e205ba5282a9fbad4a647581e23c08b821e","size_bytes":4125},{"path":"tuner/execution/providers/modal/coordinator_bundle.py","sha256":"5d8a10d2c4a0dce83174eeca202c377730820ced6945be6f7717e3a0cfd91eb0","size_bytes":31316},{"path":"tuner/execution/providers/modal/coordinator_dispatch.py","sha256":"8e5bc475c6b7d15345e22e1f8cebdabcffab5a01ffefb7d4acbfb073be02b49f","size_bytes":10571},{"path":"tuner/execution/providers/modal/coordinator_launch.py","sha256":"5975bee2d73dd09e02008ede368ebc8f03325815e8bcfd4c093736214d5f0956","size_bytes":10982},{"path":"tuner/execution/providers/modal/coordinator_preflight.py","sha256":"f452e906d37fa9dba932e9df46f829447c95ec276191e34ec3e46831c89da875","size_bytes":13746},{"path":"tuner/execution/providers/modal/coordinator_reader.py","sha256":"92a1c21725587a88435d30b6b687f63711a12ad4c77345f0776521d6503103f4","size_bytes":28524},{"path":"tuner/execution/providers/modal/coordinator_staging.py","sha256":"de47b688e859ab96a612c01b1c7869d814e40b83ae81dcd7f2b1132e18c794f0","size_bytes":20880},{"path":"tuner/execution/providers/modal/coordinator_submit_preparation.py","sha256":"28814779c7f086016ae3ff617e0f1d4cbd8b52f518d3d889a905ea03d70456ec","size_bytes":5213},{"path":"tuner/execution/providers/modal/coordinator_wire.py","sha256":"07d46b92b33e30cb45bb6b1139ce4f80afdb09e1a17a3f3c2526ddcfae6983c9","size_bytes":12998},{"path":"tuner/execution/providers/modal/deployment_identity.py","sha256":"1b7007e5a9c11abdfe0ad4f106f742faa0ee556921b4017d3b48cb772bd0da4f","size_bytes":1196},{"path":"tuner/execution/providers/modal/deployment_v1.py","sha256":"579ace8adbddfe30e09d5a39df9926a54b111a139c704fc7efc7cebf533acf94","size_bytes":3628},{"path":"tuner/execution/providers/modal/facade.py","sha256":"55b89b3d0faff6238eeead068d6963d26f27749d725d2f2b8ae32449b348e32d","size_bytes":17704},{"path":"tuner/execution/providers/modal/inference_artifacts.py","sha256":"5816f0c5662de8edad752554ba149d9527ced40ebdceeccc993c136ab099bff3","size_bytes":14973},{"path":"tuner/execution/providers/modal/inference_binding.py","sha256":"f2e8c4f1d8fc1f5e70702c88cbfff044a1f4f569e06b9a31c33a0cb9aaaf5894","size_bytes":23363},{"path":"tuner/execution/providers/modal/inference_bootstrap.py","sha256":"db2f09c1dfb2aee667b46fbb6566be8b2922d8e762896e5577d5c5078ad7d122","size_bytes":12546},{"path":"tuner/execution/providers/modal/inference_channel.py","sha256":"23492b4c7f48d86d23fb21ffd3902ac3f48bae381c30ea4759c803577911592e","size_bytes":13438},{"path":"tuner/execution/providers/modal/inference_commands.py","sha256":"1fa2dd667cb3b33e012d37b0688bfe08fc7a7cec52230a202f1f9ddda70110c5","size_bytes":4977},{"path":"tuner/execution/providers/modal/inference_entrypoint.py","sha256":"6ec8bdcbbf6d87fb5fc02a2c42d2c0f344ea965ccc4c0d94a630055c6e779991","size_bytes":10873},{"path":"tuner/execution/providers/modal/inference_model.py","sha256":"c08e1a35bb71a11c3466aedfbd8d3de87cf32d6cbd85820c3dcde2fefd14fd65","size_bytes":5935},{"path":"tuner/execution/providers/modal/inference_preparation.py","sha256":"43a1c883a8e5b2f25364e3a20f59d92a852af15fda0c9d506c213c0bf185fc32","size_bytes":49436},{"path":"tuner/execution/providers/modal/inference_runtime.py","sha256":"f8ba0d5744f90ca95e1fea3e5faf293e153a91047ffb09596f0305e1339d56ee","size_bytes":19283},{"path":"tuner/execution/providers/modal/inference_wire.py","sha256":"0492c10c6a384596bba04b3b409dc34f1f1a4afa88b7321d0d1a6fac5e29a1fb","size_bytes":20150},{"path":"tuner/execution/providers/modal/inference_worker.py","sha256":"cfb27e938c60757ef08c80b918fb522012c84918e7a0cd41a63de2cb2486cb87","size_bytes":15657},{"path":"tuner/execution/providers/modal/inference_workload.py","sha256":"4eaf399485e76829a98f57907c012658ce41d02745978827f21262ce2be94ff8","size_bytes":20421},{"path":"tuner/execution/providers/modal/manifest.py","sha256":"fc2ab4dcc46d136a681bf124d7a9fc3e5deedeeb3641d798fd207cdb62bb506f","size_bytes":8579},{"path":"tuner/execution/providers/modal/model_snapshot.py","sha256":"0b6d665c347a65150bff21b8b21c19c1f437bbcd8db69821c4240f2c57e44cdf","size_bytes":14751},{"path":"tuner/execution/providers/modal/mounted_io.py","sha256":"e49da2e749d3be1ca19fdc7f581b29fb6d1e748a86fffcf0fde2be5cdc1a0d5b","size_bytes":13076},{"path":"tuner/execution/providers/modal/prepared_input.py","sha256":"c16cef5cfcc73ab9a7826977b26e18912ac46409d91b9eae165893ed265896ac","size_bytes":8597},{"path":"tuner/execution/providers/modal/resolution.py","sha256":"5c7268aa644103ae0712b03ed55918527d6e2ed6f4120ef169866b0ffd1cbf6f","size_bytes":30179},{"path":"tuner/execution/providers/modal/runtime.py","sha256":"ada13733021a6dab19331da80076df630055abc5bddc192a34a5cda65ac718c4","size_bytes":14810},{"path":"tuner/execution/providers/modal/worker_ports.py","sha256":"adf334368dbb75e40177078cd32aedc445cd9dde8b5ece2401319d472dbccdb0","size_bytes":2928},{"path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"path":"tuner/inference/__init__.py","sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","size_bytes":0},{"path":"tuner/inference/chat_session.py","sha256":"b95a829f834406a0a0c152f9af713225fe205a9225ad7b0f959b1ba801dbcac2","size_bytes":22355},{"path":"tuner/inference/owned_process.py","sha256":"f4cbb4e1c2305615b2c127e1103f60b1e17e955fe20a2a847b1d88b7ba8c4c98","size_bytes":8832},{"path":"tuner/inference/retrieved_model.py","sha256":"4a8cf2b2cbe4f03bf7c4d240c5e59a6ec6d20826c686b3a06c0bc1642fa234d8","size_bytes":31703},{"path":"tuner/inference/run_chat.py","sha256":"9263d325224bee4b9a0e7c16045e30766d9a2e2d2b550f0508040161687d3cfc","size_bytes":6228},{"path":"tuner/inference/serving_target.py","sha256":"50ec8054d1d6d08724d7a29183ca54a04d9c915d2c04274bdac6834764bdb800","size_bytes":12227},{"path":"tuner/inference/vllm_runtime.py","sha256":"fafbcfdeee78143a2760e5fbf29621897e9be014814eefb69f1167b188c296c1","size_bytes":26019},{"path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"path":"tuner/runtime/dispatch.py","sha256":"4c41faf87d7e620bdc281490bf3725550b83dacca8b15ea47bab253f669cdd8e","size_bytes":35463},{"path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"path":"tuner/runtime/verification.py","sha256":"4c2314b20cbf7937a3f24aa90676000b9e21f0ab6701b2c1225a53e5596bf4a0","size_bytes":53856},{"path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"path":"tuner/training/contracts.py","sha256":"91c4d66d9810cc0471981c3d4ef4b54feaed5b92eb41a468f31201f8111303b5","size_bytes":21504},{"path":"tuner/training/coordinator_material.py","sha256":"c563aa3b1fbd940ddd3b8e56e4f33de70675cb831e938175f40debc0645fd9c7","size_bytes":12950},{"path":"tuner/training/grpo_compilation.py","sha256":"c6268486375bbc16f1888bb43d07b93a8de280d000a774914d59d7fd05ca5cc0","size_bytes":8998},{"path":"tuner/training/methods/__init__.py","sha256":"99db689246273707ae3df5087859ebb510e84b664f9f173c0d5827abc3e6a3ea","size_bytes":361},{"path":"tuner/training/methods/sft.py","sha256":"b501f9c214dec4b0b1550543281c25aba8dbb102604df253c641c85f7cae587b","size_bytes":9997},{"path":"tuner/training/packaged_compilation.py","sha256":"477caede6902fc587af680889b93fd9e0e42493d1a6653f3cd7413a9c7cadde8","size_bytes":9413},{"path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722},{"path":"tuner/training/post_training.py","sha256":"ba8c2ea3fe49a647d502c6a874a44658cc2e575d4cd5729966102b0ebb973a88","size_bytes":7971},{"path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"payload_bytes":1657388,"schema_version":"synaptic-modal-inference-worker-closure/v1"} diff --git a/tuner/execution/providers/modal/modal-runtime-v1.lock.json b/tuner/execution/providers/modal/modal-runtime-v1.lock.json index e27d0cca..2bc068ba 100644 --- a/tuner/execution/providers/modal/modal-runtime-v1.lock.json +++ b/tuner/execution/providers/modal/modal-runtime-v1.lock.json @@ -386,7 +386,7 @@ }, "model_preparation": { "path": "tuner/execution/providers/modal/model_snapshot.py", - "sha256": "9d4762b821b2944a8a62932d0aec105c2f6920d64bf4ceed6907e5458799e337" + "sha256": "0b6d665c347a65150bff21b8b21c19c1f437bbcd8db69821c4240f2c57e44cdf" }, "sft_runtime": { "path": "Trainers/sft/runtime_v1.py", diff --git a/tuner/execution/providers/modal/model_snapshot.py b/tuner/execution/providers/modal/model_snapshot.py index 4553e7c7..9c56862b 100644 --- a/tuner/execution/providers/modal/model_snapshot.py +++ b/tuner/execution/providers/modal/model_snapshot.py @@ -163,12 +163,7 @@ def prepare_model_snapshot( raise ModelSnapshotPreparationError(stage[0]) from None -def _prepare(*, model_ref, revision, token, persistent_root, destination_root, - scratch_root, persistent_binding, stage): - from huggingface_hub import HfApi, snapshot_download - _bind_hub_api(HfApi, snapshot_download) - - stage[0] = "INPUT" +def _validated_model_parts(model_ref, revision): parts = model_ref.split("/") if len(parts) not in (1, 2) or any( re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._-]{0,95}", part) is None @@ -176,17 +171,16 @@ def _prepare(*, model_ref, revision, token, persistent_root, destination_root, for part in parts ) or re.fullmatch(r"[0-9a-f]{40}", revision) is None: raise ValueError("model identity is not pinned") - stage[0] = "WORKSPACE_SETUP" - for root in (destination_root, scratch_root): - _directory(root) - if persistent_binding is None: - _directory(persistent_root) - credential = token if isinstance(token, str) and token.strip() else False - stage[0] = "METADATA_FETCH" - info = HfApi(endpoint=_ENDPOINT, token=credential).model_info( - model_ref, revision=revision, files_metadata=True, token=credential, - ) - stage[0] = "METADATA_VALIDATION" + return parts + + +def authoritative_model_members(info, *, model_ref, revision): + """Validate exact Hub metadata independently of retained Volume contents. + + The caller must obtain ``info`` through the pinned SDK's exact model_info + read. This pure projection is also used by ordinary model preparation. + """ + _validated_model_parts(model_ref, revision) if info.sha != revision or not info.siblings or len(info.siblings) > _MAX_FILES: raise ValueError("model metadata is not exact") members = {} @@ -219,6 +213,29 @@ def _prepare(*, model_ref, revision, token, persistent_root, destination_root, raise ValueError("model metadata digest is missing") members[name] = (size, kind, expected) + return members + + +def _prepare(*, model_ref, revision, token, persistent_root, destination_root, + scratch_root, persistent_binding, stage): + from huggingface_hub import HfApi, snapshot_download + _bind_hub_api(HfApi, snapshot_download) + + stage[0] = "INPUT" + parts = _validated_model_parts(model_ref, revision) + stage[0] = "WORKSPACE_SETUP" + for root in (destination_root, scratch_root): + _directory(root) + if persistent_binding is None: + _directory(persistent_root) + credential = token if isinstance(token, str) and token.strip() else False + stage[0] = "METADATA_FETCH" + info = HfApi(endpoint=_ENDPOINT, token=credential).model_info( + model_ref, revision=revision, files_metadata=True, token=credential, + ) + stage[0] = "METADATA_VALIDATION" + members = authoritative_model_members(info, model_ref=model_ref, revision=revision) + stage[0] = "WORKSPACE_SETUP" relative = Path("models--" + "--".join(parts)) / "snapshots" / revision destination = destination_root / "model" / relative diff --git a/tuner/runtime/manifests/grpo-constructor-library-sources-v1.json b/tuner/runtime/manifests/grpo-constructor-library-sources-v1.json new file mode 100644 index 00000000..40404b89 --- /dev/null +++ b/tuner/runtime/manifests/grpo-constructor-library-sources-v1.json @@ -0,0 +1 @@ +{"schema_version":"synaptic-grpo-constructor-library-sources/v1","sources":{"transformers/trainer.py":{"distribution":"transformers","source_sha256":"c3eac8fa28bda6330fd4dee131e7fe7d9bdc0fa35d9de944fce52cb97b379bab","version":"4.57.1","wheel_sha256":"b10d05da8fa67dc41644dbbf9bc45a44cb86ae33da6f9295f5fbf5b7890bd267"},"trl/trainer/grpo_trainer.py":{"distribution":"trl","source_sha256":"bb6905182acf2ec426f1dc9ffc37e5210d58ee47d2764f7cf12dbb90a4a28954","version":"0.24.0","wheel_sha256":"a9145b7d4a4a33778de117bda48530f0cf5b2ac25acc07db80ad04836f490dfc"},"unsloth/models/rl.py":{"distribution":"unsloth","source_sha256":"c13e9d0f10c048a4da710eb67abba9f3685277572cf730f95964289a25ac90ae","version":"2026.1.2","wheel_sha256":"1e2cab339676c99c17b7948f05199d3602119e10b66ce5479d39a54357b26a18"}}} diff --git a/tuner/runtime/manifests/offline-grpo-worker-v1.json b/tuner/runtime/manifests/offline-grpo-worker-v1.json index ef21c0e8..f1fd6e86 100644 --- a/tuner/runtime/manifests/offline-grpo-worker-v1.json +++ b/tuner/runtime/manifests/offline-grpo-worker-v1.json @@ -1 +1 @@ -{"closure_digest":"bfcc29f41d774cb393545417e5a43751776c16aa2efd9bcd39b09371d609ba87","closure_ref":"synaptic/offline-grpo-worker/v1","entrypoint":"tuner/runtime/packaged_grpo_child.py","member_count":131,"members":[{"git_mode":"100644","path":"SynthChat/__init__.py","sha256":"f9941fd4b7c15654b950b3d107ac9a0373d1ddfbe9dadcc72f8f608bfbcfa263","size_bytes":1583},{"git_mode":"100644","path":"SynthChat/config/__init__.py","sha256":"b9b4a831d9e625e63872dbd9d81fedf54064fdb2546c2cbdd3e9dde65c59a1e8","size_bytes":1162},{"git_mode":"100644","path":"SynthChat/config/config_loader.py","sha256":"09f80fd591fb9bf22afd712f613fa41d2f78b8e54f748c07d8e168082eed8fc1","size_bytes":9173},{"git_mode":"100644","path":"SynthChat/config/format_resolver.py","sha256":"264f6eeffe4068b554828e1cbc0c098eac9172dcd5fd0437c0e94a87a6c154c1","size_bytes":11651},{"git_mode":"100644","path":"SynthChat/utils/__init__.py","sha256":"0481a87cd831f9d77f6401fff01225e07ceb6295a99ce9db6391f6dfb86fc985","size_bytes":112},{"git_mode":"100644","path":"SynthChat/utils/prompt_renderer.py","sha256":"1f1479fe5ac9edeb33874422da79166abca9c582a34ebea6a532cccbd5aa15eb","size_bytes":1442},{"git_mode":"100644","path":"SynthChat/utils/yaml_loader.py","sha256":"125977de23485ea558d54744bf7bb8bb933aeae446eaae6be3f0ec09f1c8adbc","size_bytes":5264},{"git_mode":"100644","path":"Trainers/grpo/configs/config.yaml","sha256":"3346a0530959169e9d556e37b0342ba6ed93c2831164b6da225b3054d030b1e7","size_bytes":3546},{"git_mode":"100644","path":"Trainers/grpo/runtime_v1.py","sha256":"d71e2c2d68753d4ebc319ecbffd0676812278d32ad8e827e5361af1dafd94c72","size_bytes":45029},{"git_mode":"100644","path":"Trainers/grpo/src/data_loader.py","sha256":"0b9357ce4239331219b2889715c2bef6d48ef0af73bf886636eda31bca92a9ba","size_bytes":4101},{"git_mode":"100644","path":"Trainers/grpo/src/model_loader.py","sha256":"78cb7b80be05bc3f16fd05866973cd4324eff9454acea89eacb30ad37776d00a","size_bytes":7927},{"git_mode":"100644","path":"Trainers/grpo/src/rewards.py","sha256":"e157997b56cf46f0fd270d75c43c4e6b3f4b7c9f8e1a03e5e90f1cec31c18578","size_bytes":23285},{"git_mode":"100644","path":"Trainers/sft/runtime_v1.py","sha256":"460121aaee3eb83764f67d9307cfb57a3b25c117611d987577b9631b6956a1b3","size_bytes":104495},{"git_mode":"100644","path":"shared/judge/__init__.py","sha256":"08fb40e578eae3a0716dcb66664aafc9b865a02739b8777a15ac9fcd51a41848","size_bytes":870},{"git_mode":"100644","path":"shared/judge/interaction_logger.py","sha256":"41ce3ea4be06b0fdd7f2d40e5ea8f8028022fe18f912d460ae06b4c37bdb2642","size_bytes":5573},{"git_mode":"100644","path":"shared/judge/judge_service.py","sha256":"87950058f236df537d5d2921f0d9e55c4eebdc9ffe46555a3874c23d9cb1e91e","size_bytes":17998},{"git_mode":"100644","path":"shared/judge/models.py","sha256":"0c1d18ab9e69c851596d9e2e23ffe7a73cf994ee106fa70b64b8bc698de3e591","size_bytes":8459},{"git_mode":"100644","path":"shared/judge/rubric_loader.py","sha256":"84584eb9f58e8e60e78dd174412bd7d6baae96a5772b2933fe020e54364065c3","size_bytes":4191},{"git_mode":"100644","path":"shared/judge/schema_builder.py","sha256":"523b0dafc8442d92fbb68b758723067be16a698968028a76e40b470e78f73f16","size_bytes":3062},{"git_mode":"100644","path":"shared/llm/__init__.py","sha256":"44b0b934213c3f1facda937ff5fd0fd97411fd335053ba9c178e230f9b06b4ed","size_bytes":1920},{"git_mode":"100644","path":"shared/llm/base.py","sha256":"bd0cbaf0f4edbc188e511ea8fd4401cc1ec3f937681ff126c992d228009ee61e","size_bytes":9185},{"git_mode":"100644","path":"shared/llm/config.py","sha256":"a3f95c1414898860fd40bd9f2a90fc209bb1b0955d0402cb6da67bbfaf1f1420","size_bytes":9822},{"git_mode":"100644","path":"shared/llm/exceptions.py","sha256":"7223e9c03f111b10c760b4c9d7c22df5958c855a5076de0d74872d1dc6ca83f2","size_bytes":589},{"git_mode":"100644","path":"shared/llm/factory.py","sha256":"11263dcc4803ffecdbbfd343a2dbf574bfd4a4efb51f60355699be77f7871b35","size_bytes":4583},{"git_mode":"100644","path":"shared/llm/providers/__init__.py","sha256":"e0c78fec1fa6ab46068760fe2f4ef9af4d6b2645adc80fe75bbdd06039d19138","size_bytes":310},{"git_mode":"100644","path":"shared/llm/providers/lmstudio.py","sha256":"64f70a0a1ccfd67467eb026d6be495881cd53eee1fc267ee6ea059ee8b1f6282","size_bytes":7871},{"git_mode":"100644","path":"shared/llm/providers/ollama.py","sha256":"9b646ce7631dfdb6243e2660a8257d84ab2711c2fe3cb5b408466ff4f1aa780d","size_bytes":7020},{"git_mode":"100644","path":"shared/llm/providers/openai_responses.py","sha256":"70ca5fa422f1a9a29ed153609c124237c8c6d2907885972948bd00c2a9b8e623","size_bytes":10346},{"git_mode":"100644","path":"shared/llm/providers/openrouter.py","sha256":"42276fbed027f1d98183e17ad7a37e87296fe57ea73125a687cb78738dd07dda","size_bytes":17541},{"git_mode":"100644","path":"shared/llm/providers/unsloth.py","sha256":"4d60fc82e70a8f8296dcd231a68754913a3de55b8148d26de831cc177f95b137","size_bytes":11619},{"git_mode":"100644","path":"shared/llm/usage.py","sha256":"9a9d27e8689304e58d1290445bfbbe27b476b437584c6357d894b53119bd91e6","size_bytes":6964},{"git_mode":"100644","path":"shared/ml/__init__.py","sha256":"ee72d716daaba9ee99ecc8be99d776fd695436c34c5768b3e83f51f407ccf71b","size_bytes":1262},{"git_mode":"100644","path":"shared/ml/retrieval_metrics.py","sha256":"67be37fea3634cca2e7c8f42c87f2e4c207714279cece0b1fe2e9f94088ebda0","size_bytes":11456},{"git_mode":"100644","path":"shared/model_loading/__init__.py","sha256":"19e82f3112b3685a95b2f8260c5f11862d649d23968cfa6eefe306319d95a51e","size_bytes":1755},{"git_mode":"100644","path":"shared/model_loading/merge.py","sha256":"e8f0f013076b276a0f722486cb6d7583c1f9b77cde4f716a4d838ef909406711","size_bytes":9529},{"git_mode":"100644","path":"shared/training_utils.py","sha256":"15136abdb987a7bdfbc8917c49f9c77ff553e91ce53890e239ea629241345a65","size_bytes":28814},{"git_mode":"100644","path":"shared/utilities/__init__.py","sha256":"546349addf2ba6c72661e185477354d743f6a42e8ef3bf8e78ed19a17b85e2ab","size_bytes":297},{"git_mode":"100644","path":"shared/utilities/env.py","sha256":"06f56639bc506d4e497ee79d4393d17d499bfd561fd0d0679c0ea7e103b8e886","size_bytes":3318},{"git_mode":"100644","path":"shared/utilities/paths.py","sha256":"3bfa1a13e4deef02cd91a920410e839f8ab0d34ec1081903cbcb742589e995b4","size_bytes":6843},{"git_mode":"100644","path":"shared/utilities/yaml_loader.py","sha256":"203996ff297e1714b0a3e40446f719e24b6d0e20eddb91bc528100b497d8591c","size_bytes":679},{"git_mode":"100644","path":"shared/validation/__init__.py","sha256":"2c80d0a16e9b08c649b131deec3e32cfe9260d127c3b6e9c4a6508eb730e0e7f","size_bytes":1828},{"git_mode":"100644","path":"shared/validation/fitness.py","sha256":"6d90c803c59afa2e1054f33eac06c28a0d765160b53c090fd8b3bad4568cea48","size_bytes":8227},{"git_mode":"100644","path":"shared/validation/parsing/__init__.py","sha256":"e04b52ae2417aabdf8f48d80e847d60de41ec094824d4f6d088aaa132209b6c2","size_bytes":1768},{"git_mode":"100644","path":"shared/validation/parsing/configured_formats.py","sha256":"2d69eff733771bd614a8c1e1252511041fdd4bd5a13cf565205e1ebfb52dc88c","size_bytes":8162},{"git_mode":"100644","path":"shared/validation/parsing/enums.py","sha256":"4ea46d5ef556cc9d5d93d75f944e5a5120effea5264ccd89318d06228beb82bf","size_bytes":1746},{"git_mode":"100644","path":"shared/validation/parsing/response_parser.py","sha256":"5968f7d4f9f70ca269643276a71c238d6543778a47222724a1db69090b8c3471","size_bytes":21535},{"git_mode":"100644","path":"shared/validation/parsing/tool_call_parser.py","sha256":"e786758dd896bf75ceb49ea46a2a37fbea6d13692d7a1a63a26ae2872e33017a","size_bytes":13943},{"git_mode":"100644","path":"shared/validation/parsing/utilities.py","sha256":"42b1c7cb98807601a46066d5073e4d3e1984f4c848f632e46371b55e25732aa2","size_bytes":3024},{"git_mode":"100644","path":"shared/validation/rollout_filters.py","sha256":"d412c53eacb8e7fe9c459345e3dbf83a65ac50fe413d58d567d41a9c7fa6fc86","size_bytes":18795},{"git_mode":"100644","path":"shared/validation/rubric/__init__.py","sha256":"7df886540f94034002142322a481672a991d3059ec503cb996ec03e42d3123f6","size_bytes":438},{"git_mode":"100644","path":"shared/validation/rubric/rubric_cache.py","sha256":"d428466ebdadafe0c8cd8febab5e1fa49414d846efeb45c70cba47102532c2ad","size_bytes":3121},{"git_mode":"100644","path":"shared/validation/rubric/rubric_loader.py","sha256":"f77ca87d02885899d61db49fe709cba445e5c6d35fccdadb1deb8e052d8141e6","size_bytes":2313},{"git_mode":"100644","path":"shared/validation/rubric/rubric_repository.py","sha256":"1173978ad22f34625e981dc2148525c87c7745a927ee0d81c93bc1a030f56676","size_bytes":7104},{"git_mode":"100644","path":"shared/validation/validators/__init__.py","sha256":"d86436b05056c97a9b0aaae4465d23d03f407e737f4b2098fe3fc274fca64035","size_bytes":1031},{"git_mode":"100644","path":"shared/validation/validators/base.py","sha256":"764ae0d8a1439a4823a7ce82fafd1b69406873549c847b65ed57ecb3da97a264","size_bytes":1564},{"git_mode":"100644","path":"shared/validation/validators/content/__init__.py","sha256":"ed370a9fdc34b511dcfe88053bee6e9d4f30e3f7de2263cd3c675b8021865bb2","size_bytes":600},{"git_mode":"100644","path":"shared/validation/validators/content/code_validator.py","sha256":"19ded618181d16de6fa3667a64cc80a96628fd6d0f39ee5412f22c3e7aa75842","size_bytes":1493},{"git_mode":"100644","path":"shared/validation/validators/content/json_validator.py","sha256":"dd4b6fb4cb51aabaac37c052df9ff23b5051fb575dba2e893458d3d64edba497","size_bytes":7524},{"git_mode":"100644","path":"shared/validation/validators/content/regex_validator.py","sha256":"bea1124ccc79fec327a24a61e96110696c3346529d303ddff4a56ecba34b1513","size_bytes":3148},{"git_mode":"100644","path":"shared/validation/validators/content/xml_validator.py","sha256":"970937f6d8d9ef61594e2a34a2804d999482bf319c237a1891db214250bedd2e","size_bytes":1056},{"git_mode":"100644","path":"shared/validation/validators/content/yaml_validator.py","sha256":"75571df0e7ac750feeb8f92dc7da5bdc4d085e032bcf94650d826ecae07d3219","size_bytes":3738},{"git_mode":"100644","path":"shared/validation/validators/cross_scope_validator.py","sha256":"27ed2f25b96cc6fecd5e0702fdbe98a8248caf1483952a0d2402d96cf56ee269","size_bytes":8323},{"git_mode":"100644","path":"shared/validation/validators/structure_validator.py","sha256":"d637fde147e89d052229f3951ac744b56d87eb3f8553626b61dfd36bbb9829cf","size_bytes":20391},{"git_mode":"100644","path":"shared/verifiers/__init__.py","sha256":"bcc0a64a5b1be388e3dc1e913492506cac4daeb4cb5f7a210bca5bd75bafc352","size_bytes":950},{"git_mode":"100644","path":"shared/verifiers/builtins/__init__.py","sha256":"209700d4779871de25567fb6daeabad23ef3b8791e2f6c2b10749c2beead9c88","size_bytes":777},{"git_mode":"100644","path":"shared/verifiers/builtins/args_match.py","sha256":"19f44e00d8dc78d93e85361c9ecd65790fb6fa31f740f9aab743d36e20f38a0f","size_bytes":14581},{"git_mode":"100644","path":"shared/verifiers/builtins/assertion_verifier.py","sha256":"3ab2e6983adf1c9774e665961466856025fca6905d82094e2f0bc48e67cdfc46","size_bytes":17366},{"git_mode":"100644","path":"shared/verifiers/builtins/audio_verifier.py","sha256":"51c75365d411ab77535e343d771178f16fd6fee133b3d815eab0302c783fd207","size_bytes":16786},{"git_mode":"100644","path":"shared/verifiers/builtins/llm_judge.py","sha256":"b4741581e4388ed43bce5fd4def1465abc9b1eab3b8b4905d7ff07f05dd80d15","size_bytes":6915},{"git_mode":"100644","path":"shared/verifiers/builtins/retrieval_verifier.py","sha256":"b70c804369efa4a76dedeb1aeb829814952fecaecee51352803f100f12ac938d","size_bytes":15290},{"git_mode":"100644","path":"shared/verifiers/builtins/structure.py","sha256":"3030f1ca99e9355b0b37a8772937318ef334857aae5c05e92ad3db1318a4dc26","size_bytes":2167},{"git_mode":"100644","path":"shared/verifiers/builtins/substring.py","sha256":"dd86a419aa78058f5e855a538cf63e8a48c3e757f0988139bae0f231c531d76a","size_bytes":3790},{"git_mode":"100644","path":"shared/verifiers/builtins/tool_sequence.py","sha256":"111cb44e290877f1f265fd3d6d9e3d6635501b06f3648613af4f4f716b9f8d56","size_bytes":6459},{"git_mode":"100644","path":"shared/verifiers/composite.py","sha256":"678dbdb3bbab21f4a4956b7e70e8ef2d5dc883ec500a1d2278103a1f892ef66c","size_bytes":5681},{"git_mode":"100644","path":"shared/verifiers/contract.py","sha256":"c594c5527aa949d02cdf8626d03a5ef958148d647a7d7b84df6ee5c6b6ce56cc","size_bytes":2367},{"git_mode":"100644","path":"shared/verifiers/extraction.py","sha256":"f4a8e734957abb1327eeca7faa7cf5b1eb6d9247180224315e4ea6a42bc2e867","size_bytes":6056},{"git_mode":"100644","path":"shared/verifiers/registry.py","sha256":"f55035cd60ee539e4577ae1e4f101ed4b42d6c4adfb5bc0e17ae415607a3db2b","size_bytes":2475},{"git_mode":"100644","path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"git_mode":"100644","path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"git_mode":"100644","path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"git_mode":"100644","path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"git_mode":"100644","path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"git_mode":"100644","path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"git_mode":"100644","path":"synaptic_tuner/api/v1/observations.py","sha256":"b33f5826e6c03db889e0ff29c3eba65f5eab436b3d3f5146b8e4317828178d07","size_bytes":26569},{"git_mode":"100644","path":"synaptic_tuner/api/v1/planning.py","sha256":"e715b105a513fa56cfa510416e725471f8f3100cb34257be74bef3e40ca78460","size_bytes":9569},{"git_mode":"100644","path":"synaptic_tuner/api/v1/providers.py","sha256":"eca5efd5a6dee453543ae338ffddf9451d34ce1743e4ab4bde024b47be571ec0","size_bytes":4940},{"git_mode":"100644","path":"synaptic_tuner/api/v1/results.py","sha256":"65e530325c92c7afa18e3ca5f2ad9a11baf7d590221f08c33badfbf2d5f2c7b2","size_bytes":2236},{"git_mode":"100644","path":"synaptic_tuner/api/v1/training_facade.py","sha256":"bf094d0e152f5861bf8a8ff51962e0700123d9b43ed91d914b9e78bbea6facd9","size_bytes":11118},{"git_mode":"100644","path":"synaptic_tuner/api/v1/training_input.py","sha256":"209326325f8796ab7d038878e92def2c5521d7d98b7b9cc4dd1766d2e4f68c9d","size_bytes":38650},{"git_mode":"100644","path":"synaptic_tuner/api/v1/training_sources.py","sha256":"4c4ce3bc9d73e03d550c61f5ef8f18099137fe82a9be6ea96b226e7363e984b8","size_bytes":8454},{"git_mode":"100644","path":"synaptic_tuner/api/v1/usage.py","sha256":"33e36965e14ae52037f1da877c6606c46a40edf575e248769d5e5731b47b1307","size_bytes":6086},{"git_mode":"100644","path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"git_mode":"100644","path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"git_mode":"100644","path":"tuner/cloud/runtime_layout.py","sha256":"2a4cd1f5fc63d95bb3f0b5bc4f44e616bb0308d9509aceb198943953746ff705","size_bytes":9435},{"git_mode":"100644","path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"git_mode":"100644","path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"git_mode":"100644","path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"git_mode":"100644","path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"git_mode":"100644","path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"git_mode":"100644","path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"git_mode":"100644","path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"git_mode":"100644","path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"git_mode":"100644","path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"git_mode":"100644","path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"git_mode":"100644","path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"git_mode":"100644","path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"git_mode":"100644","path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"git_mode":"100644","path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"git_mode":"100644","path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"git_mode":"100644","path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"git_mode":"100644","path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"git_mode":"100644","path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"git_mode":"100644","path":"tuner/runtime/dispatch.py","sha256":"4c41faf87d7e620bdc281490bf3725550b83dacca8b15ea47bab253f669cdd8e","size_bytes":35463},{"git_mode":"100644","path":"tuner/runtime/offline_grpo_worker.py","sha256":"aac831048855e959598a4dc45b29e614f06f8714df18e868fd6b069e3c765710","size_bytes":9323},{"git_mode":"100644","path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_child.py","sha256":"124d5b2a969d38ac2e488d58a216759108f1a55dfcc3dcaf1012791efe792e4b","size_bytes":8273},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_execution.py","sha256":"a77c7d6b97ad2ded3b9631174e479e60462f1894513f90fd2fc7798828c6867d","size_bytes":14943},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_verification.py","sha256":"56564c6ee6414a3a71ce7c9ef330152d812fede2a711a2329463a0b81a61b0d6","size_bytes":10319},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_worker_closure.py","sha256":"81da8db5a61e9a874234d1e50c529e1056282fd0eef133b2720eb0f34f78cddc","size_bytes":3735},{"git_mode":"100644","path":"tuner/runtime/packaged_sft_child.py","sha256":"ab41eb7dca28b440c043f589d297dc0d9f718ed14f6cb36c96d7bd1b7bfcdc14","size_bytes":23556},{"git_mode":"100644","path":"tuner/runtime/packaged_sft_execution.py","sha256":"cab2b5032d16146422398305120f52c5a124287ecf929b640a1f17d0ea2ed648","size_bytes":80053},{"git_mode":"100644","path":"tuner/runtime/packaged_training_worker.py","sha256":"9cfe13dc9f0124b4da01d05e92cf3c2c9249d075c3a508d6b93b0a7c7f0e109f","size_bytes":34986},{"git_mode":"100644","path":"tuner/runtime/packaged_worker_closure.py","sha256":"e457937426e19747e867a314c1a2556ee64bd5cbc19359f535253b02403d1443","size_bytes":8705},{"git_mode":"100644","path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"git_mode":"100644","path":"tuner/runtime/verification.py","sha256":"4c2314b20cbf7937a3f24aa90676000b9e21f0ab6701b2c1225a53e5596bf4a0","size_bytes":53856},{"git_mode":"100644","path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"git_mode":"100644","path":"tuner/training/contracts.py","sha256":"91c4d66d9810cc0471981c3d4ef4b54feaed5b92eb41a468f31201f8111303b5","size_bytes":21504},{"git_mode":"100644","path":"tuner/training/grpo_compilation.py","sha256":"c6268486375bbc16f1888bb43d07b93a8de280d000a774914d59d7fd05ca5cc0","size_bytes":8998},{"git_mode":"100644","path":"tuner/training/packaged_compilation.py","sha256":"477caede6902fc587af680889b93fd9e0e42493d1a6653f3cd7413a9c7cadde8","size_bytes":9413},{"git_mode":"100644","path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722},{"git_mode":"100644","path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"optional_features":[],"owned_module_prefixes":["tuner","synaptic_tuner","Trainers","shared","SynthChat","Evaluator","MechInterp","configs","src"],"payload_bytes":1408447,"schema_version":"synaptic-offline-grpo-worker-closure/v1","trainer_entrypoint":"Trainers/grpo/runtime_v1.py"} +{"closure_digest":"de63243935f2f4bcec3aaf0b0ff949adff15c661d677f16eec3c70776e0fda30","closure_ref":"synaptic/offline-grpo-worker/v1","entrypoint":"tuner/runtime/packaged_grpo_child.py","member_count":133,"members":[{"git_mode":"100644","path":"SynthChat/__init__.py","sha256":"f9941fd4b7c15654b950b3d107ac9a0373d1ddfbe9dadcc72f8f608bfbcfa263","size_bytes":1583},{"git_mode":"100644","path":"SynthChat/config/__init__.py","sha256":"b9b4a831d9e625e63872dbd9d81fedf54064fdb2546c2cbdd3e9dde65c59a1e8","size_bytes":1162},{"git_mode":"100644","path":"SynthChat/config/config_loader.py","sha256":"09f80fd591fb9bf22afd712f613fa41d2f78b8e54f748c07d8e168082eed8fc1","size_bytes":9173},{"git_mode":"100644","path":"SynthChat/config/format_resolver.py","sha256":"264f6eeffe4068b554828e1cbc0c098eac9172dcd5fd0437c0e94a87a6c154c1","size_bytes":11651},{"git_mode":"100644","path":"SynthChat/utils/__init__.py","sha256":"0481a87cd831f9d77f6401fff01225e07ceb6295a99ce9db6391f6dfb86fc985","size_bytes":112},{"git_mode":"100644","path":"SynthChat/utils/prompt_renderer.py","sha256":"1f1479fe5ac9edeb33874422da79166abca9c582a34ebea6a532cccbd5aa15eb","size_bytes":1442},{"git_mode":"100644","path":"SynthChat/utils/yaml_loader.py","sha256":"125977de23485ea558d54744bf7bb8bb933aeae446eaae6be3f0ec09f1c8adbc","size_bytes":5264},{"git_mode":"100644","path":"Trainers/grpo/configs/config.yaml","sha256":"3346a0530959169e9d556e37b0342ba6ed93c2831164b6da225b3054d030b1e7","size_bytes":3546},{"git_mode":"100644","path":"Trainers/grpo/runtime_v1.py","sha256":"3cb36f79daf0cfa8dd7f400748d43faf04c22d88b34e22bd68abd5637bb32001","size_bytes":46038},{"git_mode":"100644","path":"Trainers/grpo/src/data_loader.py","sha256":"0b9357ce4239331219b2889715c2bef6d48ef0af73bf886636eda31bca92a9ba","size_bytes":4101},{"git_mode":"100644","path":"Trainers/grpo/src/model_loader.py","sha256":"78cb7b80be05bc3f16fd05866973cd4324eff9454acea89eacb30ad37776d00a","size_bytes":7927},{"git_mode":"100644","path":"Trainers/grpo/src/rewards.py","sha256":"e157997b56cf46f0fd270d75c43c4e6b3f4b7c9f8e1a03e5e90f1cec31c18578","size_bytes":23285},{"git_mode":"100644","path":"Trainers/sft/runtime_v1.py","sha256":"460121aaee3eb83764f67d9307cfb57a3b25c117611d987577b9631b6956a1b3","size_bytes":104495},{"git_mode":"100644","path":"shared/judge/__init__.py","sha256":"08fb40e578eae3a0716dcb66664aafc9b865a02739b8777a15ac9fcd51a41848","size_bytes":870},{"git_mode":"100644","path":"shared/judge/interaction_logger.py","sha256":"41ce3ea4be06b0fdd7f2d40e5ea8f8028022fe18f912d460ae06b4c37bdb2642","size_bytes":5573},{"git_mode":"100644","path":"shared/judge/judge_service.py","sha256":"87950058f236df537d5d2921f0d9e55c4eebdc9ffe46555a3874c23d9cb1e91e","size_bytes":17998},{"git_mode":"100644","path":"shared/judge/models.py","sha256":"0c1d18ab9e69c851596d9e2e23ffe7a73cf994ee106fa70b64b8bc698de3e591","size_bytes":8459},{"git_mode":"100644","path":"shared/judge/rubric_loader.py","sha256":"84584eb9f58e8e60e78dd174412bd7d6baae96a5772b2933fe020e54364065c3","size_bytes":4191},{"git_mode":"100644","path":"shared/judge/schema_builder.py","sha256":"523b0dafc8442d92fbb68b758723067be16a698968028a76e40b470e78f73f16","size_bytes":3062},{"git_mode":"100644","path":"shared/llm/__init__.py","sha256":"44b0b934213c3f1facda937ff5fd0fd97411fd335053ba9c178e230f9b06b4ed","size_bytes":1920},{"git_mode":"100644","path":"shared/llm/base.py","sha256":"bd0cbaf0f4edbc188e511ea8fd4401cc1ec3f937681ff126c992d228009ee61e","size_bytes":9185},{"git_mode":"100644","path":"shared/llm/config.py","sha256":"a3f95c1414898860fd40bd9f2a90fc209bb1b0955d0402cb6da67bbfaf1f1420","size_bytes":9822},{"git_mode":"100644","path":"shared/llm/exceptions.py","sha256":"7223e9c03f111b10c760b4c9d7c22df5958c855a5076de0d74872d1dc6ca83f2","size_bytes":589},{"git_mode":"100644","path":"shared/llm/factory.py","sha256":"11263dcc4803ffecdbbfd343a2dbf574bfd4a4efb51f60355699be77f7871b35","size_bytes":4583},{"git_mode":"100644","path":"shared/llm/providers/__init__.py","sha256":"e0c78fec1fa6ab46068760fe2f4ef9af4d6b2645adc80fe75bbdd06039d19138","size_bytes":310},{"git_mode":"100644","path":"shared/llm/providers/lmstudio.py","sha256":"64f70a0a1ccfd67467eb026d6be495881cd53eee1fc267ee6ea059ee8b1f6282","size_bytes":7871},{"git_mode":"100644","path":"shared/llm/providers/ollama.py","sha256":"9b646ce7631dfdb6243e2660a8257d84ab2711c2fe3cb5b408466ff4f1aa780d","size_bytes":7020},{"git_mode":"100644","path":"shared/llm/providers/openai_responses.py","sha256":"70ca5fa422f1a9a29ed153609c124237c8c6d2907885972948bd00c2a9b8e623","size_bytes":10346},{"git_mode":"100644","path":"shared/llm/providers/openrouter.py","sha256":"42276fbed027f1d98183e17ad7a37e87296fe57ea73125a687cb78738dd07dda","size_bytes":17541},{"git_mode":"100644","path":"shared/llm/providers/unsloth.py","sha256":"4d60fc82e70a8f8296dcd231a68754913a3de55b8148d26de831cc177f95b137","size_bytes":11619},{"git_mode":"100644","path":"shared/llm/usage.py","sha256":"9a9d27e8689304e58d1290445bfbbe27b476b437584c6357d894b53119bd91e6","size_bytes":6964},{"git_mode":"100644","path":"shared/ml/__init__.py","sha256":"ee72d716daaba9ee99ecc8be99d776fd695436c34c5768b3e83f51f407ccf71b","size_bytes":1262},{"git_mode":"100644","path":"shared/ml/retrieval_metrics.py","sha256":"67be37fea3634cca2e7c8f42c87f2e4c207714279cece0b1fe2e9f94088ebda0","size_bytes":11456},{"git_mode":"100644","path":"shared/model_loading/__init__.py","sha256":"19e82f3112b3685a95b2f8260c5f11862d649d23968cfa6eefe306319d95a51e","size_bytes":1755},{"git_mode":"100644","path":"shared/model_loading/merge.py","sha256":"e8f0f013076b276a0f722486cb6d7583c1f9b77cde4f716a4d838ef909406711","size_bytes":9529},{"git_mode":"100644","path":"shared/training_utils.py","sha256":"15136abdb987a7bdfbc8917c49f9c77ff553e91ce53890e239ea629241345a65","size_bytes":28814},{"git_mode":"100644","path":"shared/utilities/__init__.py","sha256":"546349addf2ba6c72661e185477354d743f6a42e8ef3bf8e78ed19a17b85e2ab","size_bytes":297},{"git_mode":"100644","path":"shared/utilities/env.py","sha256":"06f56639bc506d4e497ee79d4393d17d499bfd561fd0d0679c0ea7e103b8e886","size_bytes":3318},{"git_mode":"100644","path":"shared/utilities/paths.py","sha256":"3bfa1a13e4deef02cd91a920410e839f8ab0d34ec1081903cbcb742589e995b4","size_bytes":6843},{"git_mode":"100644","path":"shared/utilities/yaml_loader.py","sha256":"203996ff297e1714b0a3e40446f719e24b6d0e20eddb91bc528100b497d8591c","size_bytes":679},{"git_mode":"100644","path":"shared/validation/__init__.py","sha256":"2c80d0a16e9b08c649b131deec3e32cfe9260d127c3b6e9c4a6508eb730e0e7f","size_bytes":1828},{"git_mode":"100644","path":"shared/validation/fitness.py","sha256":"6d90c803c59afa2e1054f33eac06c28a0d765160b53c090fd8b3bad4568cea48","size_bytes":8227},{"git_mode":"100644","path":"shared/validation/parsing/__init__.py","sha256":"e04b52ae2417aabdf8f48d80e847d60de41ec094824d4f6d088aaa132209b6c2","size_bytes":1768},{"git_mode":"100644","path":"shared/validation/parsing/configured_formats.py","sha256":"2d69eff733771bd614a8c1e1252511041fdd4bd5a13cf565205e1ebfb52dc88c","size_bytes":8162},{"git_mode":"100644","path":"shared/validation/parsing/enums.py","sha256":"4ea46d5ef556cc9d5d93d75f944e5a5120effea5264ccd89318d06228beb82bf","size_bytes":1746},{"git_mode":"100644","path":"shared/validation/parsing/response_parser.py","sha256":"5968f7d4f9f70ca269643276a71c238d6543778a47222724a1db69090b8c3471","size_bytes":21535},{"git_mode":"100644","path":"shared/validation/parsing/tool_call_parser.py","sha256":"e786758dd896bf75ceb49ea46a2a37fbea6d13692d7a1a63a26ae2872e33017a","size_bytes":13943},{"git_mode":"100644","path":"shared/validation/parsing/utilities.py","sha256":"42b1c7cb98807601a46066d5073e4d3e1984f4c848f632e46371b55e25732aa2","size_bytes":3024},{"git_mode":"100644","path":"shared/validation/rollout_filters.py","sha256":"d412c53eacb8e7fe9c459345e3dbf83a65ac50fe413d58d567d41a9c7fa6fc86","size_bytes":18795},{"git_mode":"100644","path":"shared/validation/rubric/__init__.py","sha256":"7df886540f94034002142322a481672a991d3059ec503cb996ec03e42d3123f6","size_bytes":438},{"git_mode":"100644","path":"shared/validation/rubric/rubric_cache.py","sha256":"d428466ebdadafe0c8cd8febab5e1fa49414d846efeb45c70cba47102532c2ad","size_bytes":3121},{"git_mode":"100644","path":"shared/validation/rubric/rubric_loader.py","sha256":"f77ca87d02885899d61db49fe709cba445e5c6d35fccdadb1deb8e052d8141e6","size_bytes":2313},{"git_mode":"100644","path":"shared/validation/rubric/rubric_repository.py","sha256":"1173978ad22f34625e981dc2148525c87c7745a927ee0d81c93bc1a030f56676","size_bytes":7104},{"git_mode":"100644","path":"shared/validation/validators/__init__.py","sha256":"d86436b05056c97a9b0aaae4465d23d03f407e737f4b2098fe3fc274fca64035","size_bytes":1031},{"git_mode":"100644","path":"shared/validation/validators/base.py","sha256":"764ae0d8a1439a4823a7ce82fafd1b69406873549c847b65ed57ecb3da97a264","size_bytes":1564},{"git_mode":"100644","path":"shared/validation/validators/content/__init__.py","sha256":"ed370a9fdc34b511dcfe88053bee6e9d4f30e3f7de2263cd3c675b8021865bb2","size_bytes":600},{"git_mode":"100644","path":"shared/validation/validators/content/code_validator.py","sha256":"19ded618181d16de6fa3667a64cc80a96628fd6d0f39ee5412f22c3e7aa75842","size_bytes":1493},{"git_mode":"100644","path":"shared/validation/validators/content/json_validator.py","sha256":"dd4b6fb4cb51aabaac37c052df9ff23b5051fb575dba2e893458d3d64edba497","size_bytes":7524},{"git_mode":"100644","path":"shared/validation/validators/content/regex_validator.py","sha256":"bea1124ccc79fec327a24a61e96110696c3346529d303ddff4a56ecba34b1513","size_bytes":3148},{"git_mode":"100644","path":"shared/validation/validators/content/xml_validator.py","sha256":"970937f6d8d9ef61594e2a34a2804d999482bf319c237a1891db214250bedd2e","size_bytes":1056},{"git_mode":"100644","path":"shared/validation/validators/content/yaml_validator.py","sha256":"75571df0e7ac750feeb8f92dc7da5bdc4d085e032bcf94650d826ecae07d3219","size_bytes":3738},{"git_mode":"100644","path":"shared/validation/validators/cross_scope_validator.py","sha256":"27ed2f25b96cc6fecd5e0702fdbe98a8248caf1483952a0d2402d96cf56ee269","size_bytes":8323},{"git_mode":"100644","path":"shared/validation/validators/structure_validator.py","sha256":"d637fde147e89d052229f3951ac744b56d87eb3f8553626b61dfd36bbb9829cf","size_bytes":20391},{"git_mode":"100644","path":"shared/verifiers/__init__.py","sha256":"bcc0a64a5b1be388e3dc1e913492506cac4daeb4cb5f7a210bca5bd75bafc352","size_bytes":950},{"git_mode":"100644","path":"shared/verifiers/builtins/__init__.py","sha256":"209700d4779871de25567fb6daeabad23ef3b8791e2f6c2b10749c2beead9c88","size_bytes":777},{"git_mode":"100644","path":"shared/verifiers/builtins/args_match.py","sha256":"19f44e00d8dc78d93e85361c9ecd65790fb6fa31f740f9aab743d36e20f38a0f","size_bytes":14581},{"git_mode":"100644","path":"shared/verifiers/builtins/assertion_verifier.py","sha256":"3ab2e6983adf1c9774e665961466856025fca6905d82094e2f0bc48e67cdfc46","size_bytes":17366},{"git_mode":"100644","path":"shared/verifiers/builtins/audio_verifier.py","sha256":"51c75365d411ab77535e343d771178f16fd6fee133b3d815eab0302c783fd207","size_bytes":16786},{"git_mode":"100644","path":"shared/verifiers/builtins/llm_judge.py","sha256":"b4741581e4388ed43bce5fd4def1465abc9b1eab3b8b4905d7ff07f05dd80d15","size_bytes":6915},{"git_mode":"100644","path":"shared/verifiers/builtins/retrieval_verifier.py","sha256":"b70c804369efa4a76dedeb1aeb829814952fecaecee51352803f100f12ac938d","size_bytes":15290},{"git_mode":"100644","path":"shared/verifiers/builtins/structure.py","sha256":"3030f1ca99e9355b0b37a8772937318ef334857aae5c05e92ad3db1318a4dc26","size_bytes":2167},{"git_mode":"100644","path":"shared/verifiers/builtins/substring.py","sha256":"dd86a419aa78058f5e855a538cf63e8a48c3e757f0988139bae0f231c531d76a","size_bytes":3790},{"git_mode":"100644","path":"shared/verifiers/builtins/tool_sequence.py","sha256":"111cb44e290877f1f265fd3d6d9e3d6635501b06f3648613af4f4f716b9f8d56","size_bytes":6459},{"git_mode":"100644","path":"shared/verifiers/composite.py","sha256":"678dbdb3bbab21f4a4956b7e70e8ef2d5dc883ec500a1d2278103a1f892ef66c","size_bytes":5681},{"git_mode":"100644","path":"shared/verifiers/contract.py","sha256":"c594c5527aa949d02cdf8626d03a5ef958148d647a7d7b84df6ee5c6b6ce56cc","size_bytes":2367},{"git_mode":"100644","path":"shared/verifiers/extraction.py","sha256":"f4a8e734957abb1327eeca7faa7cf5b1eb6d9247180224315e4ea6a42bc2e867","size_bytes":6056},{"git_mode":"100644","path":"shared/verifiers/registry.py","sha256":"f55035cd60ee539e4577ae1e4f101ed4b42d6c4adfb5bc0e17ae415607a3db2b","size_bytes":2475},{"git_mode":"100644","path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"git_mode":"100644","path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"git_mode":"100644","path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"git_mode":"100644","path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"git_mode":"100644","path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"git_mode":"100644","path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"git_mode":"100644","path":"synaptic_tuner/api/v1/observations.py","sha256":"b33f5826e6c03db889e0ff29c3eba65f5eab436b3d3f5146b8e4317828178d07","size_bytes":26569},{"git_mode":"100644","path":"synaptic_tuner/api/v1/planning.py","sha256":"e715b105a513fa56cfa510416e725471f8f3100cb34257be74bef3e40ca78460","size_bytes":9569},{"git_mode":"100644","path":"synaptic_tuner/api/v1/providers.py","sha256":"eca5efd5a6dee453543ae338ffddf9451d34ce1743e4ab4bde024b47be571ec0","size_bytes":4940},{"git_mode":"100644","path":"synaptic_tuner/api/v1/results.py","sha256":"65e530325c92c7afa18e3ca5f2ad9a11baf7d590221f08c33badfbf2d5f2c7b2","size_bytes":2236},{"git_mode":"100644","path":"synaptic_tuner/api/v1/training_facade.py","sha256":"bf094d0e152f5861bf8a8ff51962e0700123d9b43ed91d914b9e78bbea6facd9","size_bytes":11118},{"git_mode":"100644","path":"synaptic_tuner/api/v1/training_input.py","sha256":"209326325f8796ab7d038878e92def2c5521d7d98b7b9cc4dd1766d2e4f68c9d","size_bytes":38650},{"git_mode":"100644","path":"synaptic_tuner/api/v1/training_sources.py","sha256":"4c4ce3bc9d73e03d550c61f5ef8f18099137fe82a9be6ea96b226e7363e984b8","size_bytes":8454},{"git_mode":"100644","path":"synaptic_tuner/api/v1/usage.py","sha256":"33e36965e14ae52037f1da877c6606c46a40edf575e248769d5e5731b47b1307","size_bytes":6086},{"git_mode":"100644","path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"git_mode":"100644","path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"git_mode":"100644","path":"tuner/cloud/runtime_layout.py","sha256":"2a4cd1f5fc63d95bb3f0b5bc4f44e616bb0308d9509aceb198943953746ff705","size_bytes":9435},{"git_mode":"100644","path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"git_mode":"100644","path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"git_mode":"100644","path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"git_mode":"100644","path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"git_mode":"100644","path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"git_mode":"100644","path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"git_mode":"100644","path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"git_mode":"100644","path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"git_mode":"100644","path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"git_mode":"100644","path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"git_mode":"100644","path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"git_mode":"100644","path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"git_mode":"100644","path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"git_mode":"100644","path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"git_mode":"100644","path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"git_mode":"100644","path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"git_mode":"100644","path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"git_mode":"100644","path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"git_mode":"100644","path":"tuner/runtime/dispatch.py","sha256":"4c41faf87d7e620bdc281490bf3725550b83dacca8b15ea47bab253f669cdd8e","size_bytes":35463},{"git_mode":"100644","path":"tuner/runtime/manifests/grpo-constructor-library-sources-v1.json","sha256":"e8253a7c1080ccbf02b52fa746804d9866493e88fd31fb94425f522ddd398afd","size_bytes":793},{"git_mode":"100644","path":"tuner/runtime/offline_grpo_worker.py","sha256":"0168bfc074c76051b794d5372c4d7eef2dcd52808d78a2c3e82c47b67250c19f","size_bytes":9451},{"git_mode":"100644","path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_child.py","sha256":"124d5b2a969d38ac2e488d58a216759108f1a55dfcc3dcaf1012791efe792e4b","size_bytes":8273},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_constructor_child.py","sha256":"8dc5f2908ce5defd33a178f9e24d99e24eb053e126332b3dd30e35588ce8e434","size_bytes":29218},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_execution.py","sha256":"42fd858e4b5b288ebf6492fe351537840a3a6f89f38fb6ae074fc31bc2d3f863","size_bytes":15403},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_verification.py","sha256":"56564c6ee6414a3a71ce7c9ef330152d812fede2a711a2329463a0b81a61b0d6","size_bytes":10319},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_worker_closure.py","sha256":"70baafc3eed5530b18266737ce392b6a91372115467db78d1fc9862ae770ee8b","size_bytes":3791},{"git_mode":"100644","path":"tuner/runtime/packaged_sft_child.py","sha256":"ab41eb7dca28b440c043f589d297dc0d9f718ed14f6cb36c96d7bd1b7bfcdc14","size_bytes":23556},{"git_mode":"100644","path":"tuner/runtime/packaged_sft_execution.py","sha256":"cab2b5032d16146422398305120f52c5a124287ecf929b640a1f17d0ea2ed648","size_bytes":80053},{"git_mode":"100644","path":"tuner/runtime/packaged_training_worker.py","sha256":"9cfe13dc9f0124b4da01d05e92cf3c2c9249d075c3a508d6b93b0a7c7f0e109f","size_bytes":34986},{"git_mode":"100644","path":"tuner/runtime/packaged_worker_closure.py","sha256":"e457937426e19747e867a314c1a2556ee64bd5cbc19359f535253b02403d1443","size_bytes":8705},{"git_mode":"100644","path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"git_mode":"100644","path":"tuner/runtime/verification.py","sha256":"4c2314b20cbf7937a3f24aa90676000b9e21f0ab6701b2c1225a53e5596bf4a0","size_bytes":53856},{"git_mode":"100644","path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"git_mode":"100644","path":"tuner/training/contracts.py","sha256":"91c4d66d9810cc0471981c3d4ef4b54feaed5b92eb41a468f31201f8111303b5","size_bytes":21504},{"git_mode":"100644","path":"tuner/training/grpo_compilation.py","sha256":"c6268486375bbc16f1888bb43d07b93a8de280d000a774914d59d7fd05ca5cc0","size_bytes":8998},{"git_mode":"100644","path":"tuner/training/packaged_compilation.py","sha256":"477caede6902fc587af680889b93fd9e0e42493d1a6653f3cd7413a9c7cadde8","size_bytes":9413},{"git_mode":"100644","path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722},{"git_mode":"100644","path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"optional_features":[],"owned_module_prefixes":["tuner","synaptic_tuner","Trainers","shared","SynthChat","Evaluator","MechInterp","configs","src"],"payload_bytes":1440111,"schema_version":"synaptic-offline-grpo-worker-closure/v1","trainer_entrypoint":"Trainers/grpo/runtime_v1.py"} diff --git a/tuner/runtime/manifests/packaged-grpo-worker-v1.json b/tuner/runtime/manifests/packaged-grpo-worker-v1.json index 2f1dac12..c0c78099 100644 --- a/tuner/runtime/manifests/packaged-grpo-worker-v1.json +++ b/tuner/runtime/manifests/packaged-grpo-worker-v1.json @@ -1 +1 @@ -{"closure_digest":"e48763871e7e9bc50c0c2b218bbfcb885b72b56f2ae1c2c41b1e16938c198ffe","members":[{"path":"tuner/runtime/manifests/offline-grpo-worker-v1.json","sha256":"7159247a2d15faaf2654b14168c1f90359b09950f646d0e9b2e2ec683f20617b","size_bytes":21344},{"path":"tuner/runtime/offline_grpo_worker.py","sha256":"aac831048855e959598a4dc45b29e614f06f8714df18e868fd6b069e3c765710","size_bytes":9323},{"path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"path":"tuner/runtime/packaged_grpo_child.py","sha256":"124d5b2a969d38ac2e488d58a216759108f1a55dfcc3dcaf1012791efe792e4b","size_bytes":8273},{"path":"tuner/runtime/packaged_grpo_execution.py","sha256":"a77c7d6b97ad2ded3b9631174e479e60462f1894513f90fd2fc7798828c6867d","size_bytes":14943},{"path":"tuner/runtime/packaged_grpo_verification.py","sha256":"56564c6ee6414a3a71ce7c9ef330152d812fede2a711a2329463a0b81a61b0d6","size_bytes":10319},{"path":"tuner/runtime/packaged_grpo_worker_closure.py","sha256":"81da8db5a61e9a874234d1e50c529e1056282fd0eef133b2720eb0f34f78cddc","size_bytes":3735},{"path":"tuner/runtime/packaged_sft_child.py","sha256":"ab41eb7dca28b440c043f589d297dc0d9f718ed14f6cb36c96d7bd1b7bfcdc14","size_bytes":23556},{"path":"tuner/runtime/packaged_sft_execution.py","sha256":"cab2b5032d16146422398305120f52c5a124287ecf929b640a1f17d0ea2ed648","size_bytes":80053},{"path":"tuner/runtime/packaged_training_worker.py","sha256":"9cfe13dc9f0124b4da01d05e92cf3c2c9249d075c3a508d6b93b0a7c7f0e109f","size_bytes":34986},{"path":"tuner/runtime/packaged_worker_closure.py","sha256":"e457937426e19747e867a314c1a2556ee64bd5cbc19359f535253b02403d1443","size_bytes":8705},{"path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722}],"schema_version":"synaptic-packaged-grpo-worker-closure/v1"} +{"closure_digest":"147f0ea8c3e171d8e0d97940fb539d0aff24ac211c2b64f70d9d20f418fbff45","members":[{"path":"tuner/runtime/manifests/offline-grpo-worker-v1.json","sha256":"4c07e730a3567dcdce2902b07e874b14847fd8522bb2892ae977cf759f8fbf88","size_bytes":21708},{"path":"tuner/runtime/offline_grpo_worker.py","sha256":"0168bfc074c76051b794d5372c4d7eef2dcd52808d78a2c3e82c47b67250c19f","size_bytes":9451},{"path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"path":"tuner/runtime/packaged_grpo_child.py","sha256":"124d5b2a969d38ac2e488d58a216759108f1a55dfcc3dcaf1012791efe792e4b","size_bytes":8273},{"path":"tuner/runtime/packaged_grpo_constructor_child.py","sha256":"8dc5f2908ce5defd33a178f9e24d99e24eb053e126332b3dd30e35588ce8e434","size_bytes":29218},{"path":"tuner/runtime/packaged_grpo_execution.py","sha256":"42fd858e4b5b288ebf6492fe351537840a3a6f89f38fb6ae074fc31bc2d3f863","size_bytes":15403},{"path":"tuner/runtime/packaged_grpo_verification.py","sha256":"56564c6ee6414a3a71ce7c9ef330152d812fede2a711a2329463a0b81a61b0d6","size_bytes":10319},{"path":"tuner/runtime/packaged_grpo_worker_closure.py","sha256":"70baafc3eed5530b18266737ce392b6a91372115467db78d1fc9862ae770ee8b","size_bytes":3791},{"path":"tuner/runtime/packaged_sft_child.py","sha256":"ab41eb7dca28b440c043f589d297dc0d9f718ed14f6cb36c96d7bd1b7bfcdc14","size_bytes":23556},{"path":"tuner/runtime/packaged_sft_execution.py","sha256":"cab2b5032d16146422398305120f52c5a124287ecf929b640a1f17d0ea2ed648","size_bytes":80053},{"path":"tuner/runtime/packaged_training_worker.py","sha256":"9cfe13dc9f0124b4da01d05e92cf3c2c9249d075c3a508d6b93b0a7c7f0e109f","size_bytes":34986},{"path":"tuner/runtime/packaged_worker_closure.py","sha256":"e457937426e19747e867a314c1a2556ee64bd5cbc19359f535253b02403d1443","size_bytes":8705},{"path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722}],"schema_version":"synaptic-packaged-grpo-worker-closure/v1"} diff --git a/tuner/runtime/offline_grpo_worker.py b/tuner/runtime/offline_grpo_worker.py index 3c2d76c4..aeef6bee 100644 --- a/tuner/runtime/offline_grpo_worker.py +++ b/tuner/runtime/offline_grpo_worker.py @@ -133,9 +133,11 @@ 'tuner/runtime/__init__.py', 'tuner/runtime/artifacts.py', 'tuner/runtime/dispatch.py', + 'tuner/runtime/manifests/grpo-constructor-library-sources-v1.json', 'tuner/runtime/offline_grpo_worker.py', 'tuner/runtime/offline_sft_worker.py', 'tuner/runtime/packaged_grpo_child.py', + 'tuner/runtime/packaged_grpo_constructor_child.py', 'tuner/runtime/packaged_grpo_execution.py', 'tuner/runtime/packaged_grpo_verification.py', 'tuner/runtime/packaged_grpo_worker_closure.py', diff --git a/tuner/runtime/packaged_grpo_constructor_child.py b/tuner/runtime/packaged_grpo_constructor_child.py new file mode 100644 index 00000000..72b3d765 --- /dev/null +++ b/tuner/runtime/packaged_grpo_constructor_child.py @@ -0,0 +1,601 @@ +"""Restricted, installed GRPO constructor probe. No public training authority. + +The Sandbox supervisor verifies a read-only retained snapshot and makes the +same private copy used by packaged training. Its isolated child uses the normal +GRPO environment, import guard, input builder, reward and callbacks, then stops +immediately after trainer construction. All observable diagnostics are closed. +""" +from __future__ import annotations + +import base64 +import ast +import hashlib +import importlib.resources +import inspect +import json +import os +from pathlib import Path, PurePosixPath +import re +import signal +import subprocess +import sys +import tempfile +import time + +from tuner.runtime import packaged_sft_execution as shared +from tuner.runtime.packaged_grpo_execution import grpo_child_environment +from tuner.runtime.packaged_sft_child import _installed_import_guard +from tuner.runtime.releases import parse_packaged_runtime_release + +SCHEMA = "synaptic-modal-grpo-constructor-result/v1" +DISPATCH_SCHEMA = "synaptic-modal-grpo-constructor-dispatch/v1" +_SHA = re.compile(r"[0-9a-f]{64}\Z") +_COMMIT = re.compile(r"[0-9a-f]{40}\Z") +_REF = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,95}/[A-Za-z0-9][A-Za-z0-9._-]{0,95}\Z") +_SOURCES = ("unsloth/models/rl.py", "trl/trainer/grpo_trainer.py", "transformers/trainer.py") +_STAGES = frozenset({"DISPATCH", "RELEASE", "MODEL_ADMISSION", "INPUT", "IMPORT", "REWARDS", + "DEPENDENCIES", "CONFIG", "MODEL_LIBRARY_LOAD", "LORA_ATTACH", "DATA_PREP", + "TRAINER_SETUP", "POSTCHECK"}) +_CATEGORIES = frozenset({"NONE", "TYPE", "VALUE", "OOM", "TIMEOUT", "OTHER"}) +_SITES = frozenset({"NONE", "UNSLOTH_WRAPPER", "PATCHED_GRPO_INIT", "TRANSFORMERS_BASE", "OTHER"}) + + +def canonical(value): + return json.dumps(value, ensure_ascii=True, sort_keys=True, separators=(",", ":"), + allow_nan=False).encode("ascii") + b"\n" + + +def _closed(value, allowed): + if type(value) is not str or value not in allowed: + raise ValueError + return value + + +def pinned_library_sources(): + resource = importlib.resources.files("tuner.runtime").joinpath( + "manifests", "grpo-constructor-library-sources-v1.json") + if not isinstance(resource, Path): + raise ValueError + raw = shared.stable_read(resource, 4096) + value = json.loads(raw) + if (type(value) is not dict or canonical(value) != raw + or set(value) != {"schema_version", "sources"} + or value["schema_version"] != "synaptic-grpo-constructor-library-sources/v1" + or type(value["sources"]) is not dict or set(value["sources"]) != set(_SOURCES)): + raise ValueError + versions = {"unsloth/models/rl.py": ("unsloth", "2026.1.2"), + "trl/trainer/grpo_trainer.py": ("trl", "0.24.0"), + "transformers/trainer.py": ("transformers", "4.57.1")} + for name, item in value["sources"].items(): + if (type(item) is not dict + or set(item) != {"distribution", "version", "wheel_sha256", "source_sha256"} + or (item["distribution"], item["version"]) != versions[name] + or any(type(item[key]) is not str or _SHA.fullmatch(item[key]) is None + for key in ("wheel_sha256", "source_sha256"))): + raise ValueError + return {name: value["sources"][name]["source_sha256"] for name in _SOURCES} + + +def validate_model_inventory(model): + """Expected upstream identities must come from a separate reviewed source.""" + if (type(model) is not dict or set(model) != {"ref", "revision", "tokenizer_revision", "members"} + or _REF.fullmatch(model["ref"]) is None or _COMMIT.fullmatch(model["revision"]) is None + or model["tokenizer_revision"] != model["revision"] + or type(model["members"]) is not list or not 0 < len(model["members"]) <= 64): + raise ValueError + names, total = [], 0 + for item in model["members"]: + if type(item) is not dict or set(item) != {"path", "size_bytes", "kind", "upstream_digest", "sha256"}: + raise ValueError + name = item["path"] + if (type(name) is not str or not 0 < len(name) <= 4096 or "\\" in name + or PurePosixPath(name).is_absolute() or PurePosixPath(name).as_posix() != name + or any(part in {"", ".", "..", ".git", ".cache"} for part in name.split("/"))): + raise ValueError + names.append(name) + size = item["size_bytes"] + if type(size) is not int or not 0 <= size <= 2 * 1024**3: + raise ValueError + total += size + kind = item["kind"] + expected = item["upstream_digest"] + if (kind not in {"git", "lfs"} or type(expected) is not str + or re.fullmatch(r"[0-9a-f]{64}\Z" if kind == "lfs" else r"[0-9a-f]{40}\Z", expected) is None + or type(item["sha256"]) is not str or _SHA.fullmatch(item["sha256"]) is None): + raise ValueError + if names != sorted(set(names)) or total > 2 * 1024**3: + raise ValueError + return tuple((item["path"], item["size_bytes"], item["sha256"]) for item in model["members"]) + + +def validate_dispatch(raw): + if type(raw) is not bytes or not 0 < len(raw) <= 131072: + raise ValueError + value = json.loads(raw) + if (type(value) is not dict or canonical(value) != raw + or set(value) != {"schema_version", "attempt_ref", "source_commit", + "host_source_commit", "engine_gitlink", "release", + "deployment_facts_digest", "image_id", "input_base64", "input_sha256", + "model", "library_sources"} + or value["schema_version"] != DISPATCH_SCHEMA + or type(value["attempt_ref"]) is not str or re.fullmatch(r"grpo-diag-[0-9a-f]{24}\Z", value["attempt_ref"]) is None + or type(value["source_commit"]) is not str or _COMMIT.fullmatch(value["source_commit"]) is None + or type(value["host_source_commit"]) is not str or _COMMIT.fullmatch(value["host_source_commit"]) is None + or value["engine_gitlink"] != value["source_commit"] + or type(value["deployment_facts_digest"]) is not str or _SHA.fullmatch(value["deployment_facts_digest"]) is None + or type(value["image_id"]) is not str or re.fullmatch(r"im-[A-Za-z0-9]{1,64}\Z", value["image_id"]) is None + or type(value["input_sha256"]) is not str or _SHA.fullmatch(value["input_sha256"]) is None): + raise ValueError + validate_model_inventory(value["model"]) + sources = value["library_sources"] + if type(sources) is not dict or sources != pinned_library_sources(): + raise ValueError + input_raw = base64.b64decode(value["input_base64"], validate=True) + if hashlib.sha256(input_raw).hexdigest() != value["input_sha256"]: + raise ValueError + verify_probe_input(input_raw) + parse_packaged_runtime_release(value["release"]) + return value, input_raw + + +def verify_probe_input(raw): + if type(raw) is not bytes or not 0 < len(raw) <= 65536: + raise ValueError + document = json.loads(raw) + if (type(document) is not dict or canonical(document) != raw + or set(document) != {"schema_version", "workload", "dataset_base64", + "qualification_policy", "row_count", "runtime_profile", + "profile_sha256", "recipe_sha256", "plan_sha256", + "dataset_sha256"} + or document["schema_version"] != "synaptic-grpo-constructor-probe-input/v3" + or document["row_count"] != 4): + raise ValueError + dataset = base64.b64decode(document["dataset_base64"], validate=True) + if (type(document["runtime_profile"]) is not str + or re.fullmatch(r"[a-z0-9][a-z0-9_-]{0,95}\Z", document["runtime_profile"]) is None + or type(document["profile_sha256"]) is not str + or re.fullmatch(r"sha256:[0-9a-f]{64}\Z", document["profile_sha256"]) is None + or any(type(document[key]) is not str or _SHA.fullmatch(document[key]) is None + for key in ("recipe_sha256", "plan_sha256", "dataset_sha256")) + or hashlib.sha256(dataset).hexdigest() != document["dataset_sha256"]): + raise ValueError + workload = document["workload"] + from Trainers.grpo.runtime_v1 import admit_grpo_inputs + config, rows, documents, schema = admit_grpo_inputs( + config=workload["config"], dataset_bytes=dataset, + reward_documents=workload["reward_documents"], + reward_schema=workload["reward_schema"], + model_identity=workload["model"], workload=workload) + if (len(rows) != 4 or config != workload["config"] + or documents != workload["reward_documents"] + or schema != workload["reward_schema"]): + raise ValueError + from tuner.training.grpo_compilation import validate_qualification_evidence_policy + validate_qualification_evidence_policy(document["qualification_policy"]) + return document + + +def _source_verified(sources): + """Check installed source identities before projecting a library region.""" + import importlib.metadata + import importlib.util + for distribution, version in (("unsloth", "2026.1.2"), ("trl", "0.24.0"), + ("transformers", "4.57.1")): + if importlib.metadata.version(distribution) != version: + return False + for name in _SOURCES: + module = name[:-3].replace("/", ".") + spec = importlib.util.find_spec(module) + if spec is None or type(spec.origin) is not str: + return False + path = Path(spec.origin) + if not path.is_absolute() or path.is_symlink(): + return False + if hashlib.sha256(shared.stable_read(path, 4 * 1024 * 1024)).hexdigest() != sources[name]: + return False + return True + + +def _code(method): + return getattr(method, "__code__", None) + + +def _site(error, trainer_class, sources_verified): + if not sources_verified: + return "OTHER" + # Unsloth 2026.1.2 writes a generated UnslothGRPOTrainer.py at import + # time. Its bytes are not covered by the three pinned wheel-source hashes. + # MRO position or a generated filename cannot authenticate that code. + try: + import transformers + from trl import GRPOTrainer + if trainer_class is not GRPOTrainer: + return "OTHER" + if transformers.Trainer not in trainer_class.__mro__: + return "OTHER" + base = _code(transformers.Trainer.__init__) + if base is None: + return "OTHER" + from importlib.util import find_spec + origin = find_spec("transformers.trainer").origin + if not isinstance(origin, str) or Path(base.co_filename).resolve() != Path(origin).resolve(): + return "OTHER" + source = shared.stable_read(Path(origin), 2 * 1024 * 1024) + if hashlib.sha256(source).hexdigest() != pinned_library_sources()["transformers/trainer.py"]: + return "OTHER" + parsed = ast.parse(source) + methods = [method for node in parsed.body if isinstance(node, ast.ClassDef) + and node.name == "Trainer" for method in node.body + if isinstance(method, (ast.FunctionDef, ast.AsyncFunctionDef)) + and method.name == "__init__"] + if len(methods) != 1 or base.co_firstlineno != min( + [methods[0].lineno, *(part.lineno for part in methods[0].decorator_list)]): + return "OTHER" + except BaseException: + return "OTHER" + tb = error.__traceback__ + while tb is not None: + if tb.tb_frame.f_code is base: + return "TRANSFORMERS_BASE" + tb = tb.tb_next + return "OTHER" + + +def _dtype_relation(model, args): + try: + import torch + dtype = getattr(model.config, "dtype", None) or getattr(model.config, "torch_dtype", None) + if dtype is None: + dtype = model.get_input_embeddings().weight.dtype + fp16, bf16 = getattr(args, "fp16", None), getattr(args, "bf16", None) + if type(fp16) is not bool or type(bf16) is not bool: + return "UNKNOWN" + if dtype == torch.float16 and bf16: + return "MISMATCH" + if isinstance(dtype, torch.dtype) and dtype != torch.float16 and fp16: + return "MISMATCH" + if (dtype == torch.float16 and fp16 and not bf16) or (dtype == torch.bfloat16 and bf16 and not fp16): + return "MATCH" + except BaseException: + pass + return "UNKNOWN" + + +def _observation(trainer_class, kwargs, verified): + result = {"patched_class": "UNKNOWN", "signature_binding": "UNAVAILABLE", + "precision_relation": "UNKNOWN", "model_predicate": "OTHER", + "tokenizer_predicate": "OTHER", "reward_predicate": "OTHER", + "callbacks_predicate": "OTHER", "dataset_predicate": "OTHER", + "class_digest": None, "signature_digest": None} + try: + from transformers import PreTrainedTokenizerBase, TrainerCallback + from datasets import Dataset + signature = inspect.signature(trainer_class.__init__) + result["signature_digest"] = hashlib.sha256(str(signature).encode("utf-8")).hexdigest() + result["class_digest"] = hashlib.sha256(canonical((trainer_class.__module__, trainer_class.__qualname__))).hexdigest() + try: + signature.bind(object(), **kwargs) + result["signature_binding"] = "ACCEPTED" + except TypeError: + result["signature_binding"] = "REJECTED" + chain = trainer_class.__mro__ + if verified: + from trl import GRPOTrainer + if trainer_class is GRPOTrainer and len(chain) > 1: + # Unsloth compiles the outer class at import time. The pinned + # wheel source does not authenticate that generated code. + result["patched_class"] = ("UNKNOWN" if + (chain[0].__name__, chain[1].__name__) == + ("UnslothGRPOTrainer", "_UnslothGRPOTrainer") else "NO") + model, tokenizer, reward, callbacks, data = (kwargs[key] for key in + ("model", "processing_class", "reward_funcs", "callbacks", "train_dataset")) + result["precision_relation"] = _dtype_relation(model, kwargs["args"]) + result["model_predicate"] = "HAS_CONFIG_EMBEDDINGS" if hasattr(model, "config") and callable(getattr(model, "get_input_embeddings", None)) else "OTHER" + result["tokenizer_predicate"] = "PRETRAINED_TOKENIZER" if isinstance(tokenizer, PreTrainedTokenizerBase) else "OTHER" + result["reward_predicate"] = "CALLABLE_NAMED" if callable(reward) and type(getattr(reward, "__name__", None)) is str else "OTHER" + result["callbacks_predicate"] = "TRAINER_CALLBACKS" if type(callbacks) is list and len(callbacks) == 2 and all(isinstance(value, TrainerCallback) for value in callbacks) else "OTHER" + result["dataset_predicate"] = "DATASET_FOUR" if isinstance(data, Dataset) and len(data) == 4 else "OTHER" + except BaseException: + pass + return result + + +def _empty_result(): + return {"schema_version": SCHEMA, "status": "REJECTED", "stage": "DISPATCH", + "category": "OTHER", "type_site": "OTHER", "source_verified": False, + "patched_class": "UNKNOWN", "signature_binding": "UNAVAILABLE", + "precision_relation": "UNKNOWN", "model_predicate": "OTHER", + "tokenizer_predicate": "OTHER", "reward_predicate": "OTHER", + "callbacks_predicate": "OTHER", "dataset_predicate": "OTHER", + "class_digest": None, "signature_digest": None, + "training_executed": False, "generation_executed": False, + "optimizer_update_executed": False} + + +def validate_result(raw): + if type(raw) is not bytes or not 0 < len(raw) <= 8192: + raise ValueError + value = json.loads(raw) + expected = _empty_result() + if type(value) is not dict or set(value) != set(expected) or canonical(value) != raw: + raise ValueError + _closed(value["status"], {"REJECTED", "CONSTRUCTED"}) + _closed(value["stage"], _STAGES) + _closed(value["category"], _CATEGORIES) + _closed(value["type_site"], _SITES) + for key, allowed in (("patched_class", {"YES", "NO", "UNKNOWN"}), + ("signature_binding", {"ACCEPTED", "REJECTED", "UNAVAILABLE"}), + ("precision_relation", {"MATCH", "MISMATCH", "UNKNOWN"}), + ("model_predicate", {"HAS_CONFIG_EMBEDDINGS", "OTHER"}), + ("tokenizer_predicate", {"PRETRAINED_TOKENIZER", "OTHER"}), + ("reward_predicate", {"CALLABLE_NAMED", "OTHER"}), + ("callbacks_predicate", {"TRAINER_CALLBACKS", "OTHER"}), + ("dataset_predicate", {"DATASET_FOUR", "OTHER"})): + _closed(value[key], allowed) + if (type(value["source_verified"]) is not bool + or any(value[key] is not False for key in + ("training_executed", "generation_executed", "optimizer_update_executed")) + or any(item is not None and (type(item) is not str or _SHA.fullmatch(item) is None) + for item in (value["class_digest"], value["signature_digest"]))): + raise ValueError + if value["status"] == "CONSTRUCTED": + if (value["stage"] != "TRAINER_SETUP" or value["category"] != "NONE" + or value["type_site"] != "NONE" or value["source_verified"] is not True): + raise ValueError + elif value["category"] == "NONE": + raise ValueError + return value + + +def _result_from_error(result, error, stage, trainer_class, verified): + result["status"] = "REJECTED" + result["stage"] = stage if stage in _STAGES else "POSTCHECK" + result["category"] = ("TYPE" if isinstance(error, TypeError) else + "VALUE" if isinstance(error, ValueError) else + "TIMEOUT" if isinstance(error, TimeoutError) else + "OOM" if isinstance(error, MemoryError) else "OTHER") + result["type_site"] = _site(error, trainer_class, verified) if result["category"] == "TYPE" and trainer_class is not None else "OTHER" + return result + + +def _isolated(transport, digest, result_fd): + result, stage, trainer_class, verified = _empty_result(), "DISPATCH", None, False + retained = None + try: + raw = shared.stable_read(Path(transport), 131072) + if hashlib.sha256(raw).hexdigest() != digest: + raise ValueError + value = json.loads(raw) + if (type(value) is not dict or canonical(value) != raw + or set(value) != {"dispatch", "private_manifest", "paths", "environment", "deadline"}): + raise ValueError + dispatch, probe_bytes = validate_dispatch(canonical(value["dispatch"])) + release = parse_packaged_runtime_release(dispatch["release"]) + paths = shared.PackagedSFTPaths(**{key: Path(item) for key, item in value["paths"].items()}) + environment = grpo_child_environment(release, paths, Path(os.environ.get("SYNAPTIC_MODEL_SNAPSHOT", ""))) + if (dict(os.environ) != dict(environment) or value["environment"] != list(map(list, environment)) + or not sys.flags.isolated or Path.cwd() != paths.tmp + or time.monotonic() >= value["deadline"]): + raise ValueError + stage = "RELEASE" + trainer_source = shared._inspect_release(release, method="grpo") + guard = _installed_import_guard(release, trainer_source, method="grpo") + sys.meta_path.insert(0, guard) + try: + stage = "MODEL_ADMISSION" + model = dispatch["model"] + retained = shared._retain_private_snapshot(value["private_manifest"], paths, + {"ref": model["ref"], "revision": model["revision"], "tokenizer_revision": model["tokenizer_revision"]}) + retained.check() + stage = "INPUT" + probe = verify_probe_input(probe_bytes) + dataset = base64.b64decode(probe["dataset_base64"], validate=True) + if hashlib.sha256(dataset).hexdigest() != probe["dataset_sha256"]: + raise ValueError + stage = "IMPORT" + from Trainers.grpo.runtime_v1 import run_grpo + output = paths.state / "grpo-output" + output.mkdir(mode=0o700, exist_ok=False) + def mark(value): + nonlocal stage + if value in _STAGES: + stage = value + def observe(cls, kwargs): + nonlocal trainer_class, verified + trainer_class = cls + verified = _source_verified(dispatch["library_sources"]) + result["source_verified"] = verified + if not verified: + raise ValueError + result.update(_observation(cls, kwargs, verified)) + trainer = run_grpo( + config=probe["workload"]["config"], dataset_bytes=dataset, + reward_documents=probe["workload"]["reward_documents"], + reward_schema=probe["workload"]["reward_schema"], + model_snapshot=retained.snapshot, + model_identity={"model_id": model["ref"], "revision": model["revision"], + "tokenizer_revision": model["tokenizer_revision"]}, + output_root=output, workload=probe["workload"], + metric_sink=lambda record: None, _packaged_artifacts=True, + qualification_evidence=probe["qualification_policy"], + _child_deadline=value["deadline"], _runtime_milestone=mark, + _constructor_only=True, _constructor_observer=observe) + del trainer + retained.check() + if time.monotonic() >= value["deadline"]: + raise TimeoutError + result.update(status="CONSTRUCTED", stage="TRAINER_SETUP", category="NONE", type_site="NONE") + finally: + sys.meta_path.remove(guard) + except BaseException as error: + _result_from_error(result, error, stage, trainer_class, verified) + finally: + if retained is not None: + try: + retained.close() + except BaseException: + result.update(status="REJECTED", stage="POSTCHECK", category="OTHER", type_site="OTHER") + payload = canonical(result) + if len(payload) > 8192: + payload = canonical(_empty_result()) + os.write(result_fd, payload) + + +def _private_paths(root, cache): + for name in ("state", "tmp", "tracking", "artifacts", "prepared_input"): + (root / name).mkdir(mode=0o700) + return shared.PackagedSFTPaths(root / "prepared_input", root / "artifacts", + root / "state", root / "tracking", cache, root / "tmp") + + +def _verify_retained_model(model, cache): + from tuner.execution.providers.modal.model_snapshot import _verify + relative = Path("models--" + model["ref"].replace("/", "--")) / "snapshots" / model["revision"] + snapshot = cache / relative + held, current = shared._snapshot_inventory(snapshot, cache) + expected = validate_model_inventory(model) + try: + if current != expected: + raise ValueError + for item in model["members"]: + if _verify(snapshot / item["path"], item["size_bytes"], item["kind"], + item["upstream_digest"]) != item["sha256"]: + raise ValueError + for directory in held: + directory.check() + return snapshot, held, current + except BaseException: + shared._close_resources(held) + raise + + +def _wait_isolated_child(argv, *, cwd, environment, result_fd, deadline): + """Own and reap the entire child process group under one finite deadline.""" + child = subprocess.Popen(argv, cwd=cwd, env=environment, pass_fds=(result_fd,), + stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, + start_new_session=True) + try: + child.wait(timeout=max(0.1, deadline - time.monotonic())) + except BaseException as error: + try: + os.killpg(child.pid, signal.SIGKILL) + except ProcessLookupError: + pass + try: + child.wait(timeout=5) + except BaseException: + pass + if isinstance(error, subprocess.TimeoutExpired): + raise TimeoutError from None + raise + return child.returncode + + +_RESULT_PATH = "/tmp/synaptic-grpo-constructor-result.json" +_READY_PATH = "/tmp/synaptic-grpo-constructor-ready" +_ACK_PATH = "/tmp/synaptic-grpo-constructor-ack" + + +def _supervise(): + result = _empty_result() + result_fd = os.open(_RESULT_PATH, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o600) + sink = os.open(os.devnull, os.O_WRONLY) + os.dup2(sink, 1) + os.dup2(sink, 2) + os.close(sink) + held, private = (), None + try: + raw = sys.stdin.buffer.read(131073) + dispatch, _ = validate_dispatch(raw) + stage = "RELEASE" + release = parse_packaged_runtime_release(dispatch["release"]) + shared._inspect_release(release, method="grpo") + cache = Path("/mnt/synaptic-model-cache") + stage = "MODEL_ADMISSION" + snapshot, held, inventory = _verify_retained_model(dispatch["model"], cache) + with tempfile.TemporaryDirectory(prefix="synaptic-grpo-diagnostic-") as temporary: + root = Path(temporary) + paths = _private_paths(root, cache) + model = dispatch["model"] + private = shared._copy_private_snapshot(snapshot, held, inventory, paths, + {"ref": model["ref"], "revision": model["revision"], "tokenizer_revision": model["tokenizer_revision"]}) + private.check() + stage = "INPUT" + deadline = time.monotonic() + 110 + environment = grpo_child_environment(release, paths, private.snapshot) + transport = canonical({"dispatch": dispatch, "private_manifest": private.manifest, + "paths": {key: str(getattr(paths, key)) for key in paths.__dataclass_fields__}, + "environment": list(map(list, environment)), "deadline": deadline}) + target = paths.state / "constructor-transport.json" + with target.open("xb") as stream: + stream.write(transport) + stream.flush() + os.fsync(stream.fileno()) + answer = paths.state / "constructor-result.json" + answer_fd = os.open(answer, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o600) + try: + exit_code = _wait_isolated_child((release.python_executable, "-I", "-B", "-m", + "tuner.runtime.packaged_grpo_constructor_child", "--isolated", str(target), + hashlib.sha256(transport).hexdigest(), str(answer_fd)), + cwd=paths.tmp, environment=dict(environment), result_fd=answer_fd, + deadline=deadline) + if exit_code != 0: + raise ValueError + finally: + os.close(answer_fd) + response = shared.stable_read(answer, 8192) + result = validate_result(response) + private.check() + for directory in held: + directory.check() + except BaseException as error: + stage = locals().get("stage", "DISPATCH") + _result_from_error(result, error, stage, None, False) + finally: + if private is not None: + try: + private.close() + except BaseException: + result.update(status="REJECTED", stage="POSTCHECK", category="OTHER", type_site="OTHER") + if held: + if shared._close_resources(held): + result.update(status="REJECTED", stage="POSTCHECK", category="OTHER", type_site="OTHER") + payload = canonical(result) + if len(payload) > 8192: + payload = canonical(_empty_result()) + os.write(result_fd, payload) + os.close(result_fd) + with open(_READY_PATH, "xb") as marker: + marker.write(b"1") + marker.flush() + os.fsync(marker.fileno()) + # Modal's Sandbox filesystem is only live while this process runs. + # Keep it available for a bounded host read, then exit on one ack. + until = time.monotonic() + 10 + while time.monotonic() < until: + try: + with open(_ACK_PATH, "rb") as marker: + if marker.read(2) == b"1": + break + except FileNotFoundError: + pass + time.sleep(0.05) + return 0 + + +def main(): + if sys.argv[1:] == ["--supervise"]: + return _supervise() + if len(sys.argv) == 5 and sys.argv[1] == "--isolated" and sys.flags.isolated: + result_fd = int(sys.argv[4]) + sink = os.open(os.devnull, os.O_WRONLY) + os.dup2(sink, 1) + os.dup2(sink, 2) + os.close(sink) + _isolated(sys.argv[2], sys.argv[3], result_fd) + return 0 + return 125 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tuner/runtime/packaged_grpo_execution.py b/tuner/runtime/packaged_grpo_execution.py index a01fc265..7efeb95e 100644 --- a/tuner/runtime/packaged_grpo_execution.py +++ b/tuner/runtime/packaged_grpo_execution.py @@ -161,20 +161,29 @@ def admit_packaged_grpo(*, runtime_release, provider_binding, execution_binding, raise PackagedGRPOExecutionError("ADMISSION") from None -def _invocation_spec(admitted, manifest, dataset_fd, metric_fd, deadline): - paths = admitted.paths - model = _model(parse_packaged_grpo_workload(admitted.workload_bytes)) - _, snapshot = shared._private_manifest(manifest, paths, model) - environment = dict(admitted.environment) +def grpo_child_environment(release, paths, snapshot, declared=()): + """One replacement environment for normal and constructor-only GRPO children.""" + environment = dict(declared) + if (len(environment) != len(declared) + or not set(environment) <= {"CUDA_VISIBLE_DEVICES", "NVIDIA_VISIBLE_DEVICES", "LANG", "LC_ALL"}): + raise ValueError environment.setdefault("LC_ALL", "C.UTF-8") environment.update({ - "PATH": ":".join(dict.fromkeys((str(Path(admitted.release.python_executable).parent), + "PATH": ":".join(dict.fromkeys((str(Path(release.python_executable).parent), "/usr/local/bin", "/usr/bin", "/bin"))), "PYTHONNOUSERSITE": "1", "PYTHONSAFEPATH": "1", "PYTHONDONTWRITEBYTECODE": "1", "HF_HOME": str(paths.cache / "huggingface"), "TRANSFORMERS_CACHE": str(paths.cache / "transformers"), "HF_HUB_OFFLINE": "1", "TRANSFORMERS_OFFLINE": "1", "WANDB_DISABLED": "true", "TMPDIR": str(paths.tmp), "HOME": str(paths.tmp), "SYNAPTIC_MODEL_SNAPSHOT": str(snapshot), }) + return tuple(sorted(environment.items())) + + +def _invocation_spec(admitted, manifest, dataset_fd, metric_fd, deadline): + paths = admitted.paths + model = _model(parse_packaged_grpo_workload(admitted.workload_bytes)) + _, snapshot = shared._private_manifest(manifest, paths, model) + environment = dict(grpo_child_environment(admitted.release, paths, snapshot, admitted.environment)) payload = { "schema_version": _CHILD_SCHEMA, "release": admitted.release.to_dict(), "provider_binding": admitted.provider_binding.to_dict(), "execution": admitted.execution.to_dict(), diff --git a/tuner/runtime/packaged_grpo_worker_closure.py b/tuner/runtime/packaged_grpo_worker_closure.py index 2aa1635e..a2f0dc54 100644 --- a/tuner/runtime/packaged_grpo_worker_closure.py +++ b/tuner/runtime/packaged_grpo_worker_closure.py @@ -13,7 +13,8 @@ GRPO_OUTER_SCHEMA = "synaptic-packaged-grpo-worker-closure/v1" GRPO_OUTER_RESOURCE = "manifests/packaged-grpo-worker-v1.json" GRPO_OUTER_MEMBERS = tuple(sorted(( - "tuner/runtime/packaged_grpo_child.py", "tuner/runtime/packaged_grpo_execution.py", + "tuner/runtime/packaged_grpo_child.py", "tuner/runtime/packaged_grpo_constructor_child.py", + "tuner/runtime/packaged_grpo_execution.py", "tuner/runtime/packaged_grpo_verification.py", "tuner/runtime/packaged_grpo_worker_closure.py", "tuner/runtime/packaged_sft_child.py", "tuner/runtime/packaged_sft_execution.py", "tuner/runtime/packaged_training_worker.py", "tuner/runtime/packaged_worker_closure.py", diff --git a/tuner/training/modal_grpo_constructor_diagnostic.py b/tuner/training/modal_grpo_constructor_diagnostic.py new file mode 100644 index 00000000..b8f3ef52 --- /dev/null +++ b/tuner/training/modal_grpo_constructor_diagnostic.py @@ -0,0 +1,641 @@ +"""Restricted one-shot Modal GRPO constructor capture beside CPU qualification. + +This private operator shares the packaged deployment, source, attempt journal, +catalog and bounded Sandbox capture boundaries. Its result cannot be parsed as +a CPU or training qualification receipt and grants no TrainingAPI authority. +""" +from __future__ import annotations + +import base64 +from dataclasses import dataclass, field +import hashlib +from io import BytesIO +from pathlib import Path +import signal +import ssl +import threading +import time +from types import SimpleNamespace +from urllib.parse import quote, urlparse +from urllib.request import HTTPSHandler, HTTPRedirectHandler, ProxyHandler, Request, build_opener +from zipfile import ZipFile + +from scripts.capture_modal_inference_runtime import _OperationOwnership, _bounded_call +from tuner.execution.foundation_v2.canonical import canonical_bytes, parse_canonical_object +from tuner.execution.providers.modal.facade import MODAL_VOLUME_V1 +from tuner.execution.providers.modal.model_snapshot import authoritative_model_members +from tuner.execution.providers.modal.runtime_release_qualification import ( + QUALIFICATION_HMAC_KEY_REF, ModalRuntimeQualificationHmacAuthenticator, + _observe_current_layout, _validate_deployment_release, +) +from tuner.project.source_bundle import inspect_git_source, _git +from tuner.runtime.packaged_grpo_constructor_child import ( + DISPATCH_SCHEMA, canonical, validate_dispatch, validate_model_inventory, validate_result, +) +from tuner.runtime.packaged_grpo_worker_closure import load_packaged_grpo_worker_closure +from tuner.runtime.offline_grpo_worker import load_packaged_offline_grpo_worker_manifest +from tuner.runtime.packaged_worker_closure import stable_read +from tuner.training.modal_host_qualification import _CurrentQualificationObserver +from tuner.training.modal_host_runtime import ModalHostRuntimeV1, quote_modal_runtime_for_host + +_SEAL = object() +_PURPOSE = "modal-grpo-constructor-diagnostic/v1" +_CLAIM = "synaptic-modal-grpo-constructor-claim/v1" +_RECEIPT = "synaptic-modal-grpo-constructor-receipt/v1" +_OUTPUT_MAX = 8192 + + +class ModalGRPOConstructorUnavailable(RuntimeError): + """Closed diagnostic failure; claims remain consumed on uncertain effects.""" + + def __init__(self, phase): + if phase not in {"ADMISSION", "CLAIM", "QUOTE", "APP", "LAYOUT", "VOLUME", "IMAGE", + "CREATE", "INPUT", "OUTPUT", "TERMINAL", "CLEANUP", "RECEIPT"}: + raise ValueError + self.phase = phase + super().__init__("modal_grpo_constructor_diagnostic_unavailable") + + +@dataclass(frozen=True, slots=True) +class ModalGRPOConstructorAdmission: + dispatch_bytes: bytes + attempt_ref: str + source_commit: str + host_source_commit: str + wheel_sha256: str + runtime_release_digest: str + deployment_facts_digest: str + image_id: str + snapshot_deployment_facts_digest: str + model_cache_volume_id: str + model_cache_volume_name: str + quote_digest: str + _seal: object = field(repr=False) + + +def _model_role(runtime): + matches = tuple(item.volume_id for item in runtime.deployment_facts.volumes + if item.spec.role == "model_cache") + if len(matches) != 1 or matches[0] != runtime.facts.model_cache_volume_id: + raise ValueError + return matches[0] + + +_METADATA_MAX = 1 << 20 +_GIT_FILES_MAX = 32 << 20 + + +class _HubOnlyRedirect(HTTPRedirectHandler): + def redirect_request(self, request, fp, code, message, headers, new_url): + parsed = urlparse(new_url) + if parsed.scheme != "https" or parsed.hostname != "huggingface.co": + raise ValueError + return super().redirect_request(request, fp, code, message, headers, new_url) + + +def _bounded_hub_bytes(path, *, maximum, deadline, opener=None): + """Exact anonymous Hub read with a whole-operation timer and byte cap.""" + if (threading.current_thread() is not threading.main_thread() + or not hasattr(signal, "setitimer") or type(path) is not str + or not path.startswith("https://huggingface.co/") + or type(maximum) is not int or not 0 < maximum <= _GIT_FILES_MAX + or time.monotonic() >= deadline): + raise ValueError + selected = opener or build_opener(ProxyHandler({}), HTTPSHandler(context=ssl.create_default_context()), + _HubOnlyRedirect()) + old_handler = signal.getsignal(signal.SIGALRM) + old_timer = signal.getitimer(signal.ITIMER_REAL) + if old_timer != (0.0, 0.0): + raise ValueError + def expired(signum, frame): + raise TimeoutError + signal.signal(signal.SIGALRM, expired) + signal.setitimer(signal.ITIMER_REAL, max(0.001, deadline - time.monotonic())) + try: + request = Request(path, headers={"Accept-Encoding": "identity", "Accept": "application/json"}) + with selected.open(request, timeout=max(0.001, min(5.0, deadline - time.monotonic()))) as response: + parsed = urlparse(response.geturl()) + if (parsed.scheme != "https" or parsed.hostname != "huggingface.co" + or response.headers.get("Content-Encoding", "identity") != "identity"): + raise ValueError + length = response.headers.get("Content-Length") + if length is not None and (not length.isdecimal() or int(length) > maximum): + raise ValueError + data = bytearray() + while True: + if time.monotonic() >= deadline: + raise TimeoutError + chunk = response.read(min(65536, maximum + 1 - len(data))) + if not chunk: + break + data.extend(chunk) + if len(data) > maximum: + raise ValueError + return bytes(data) + finally: + signal.setitimer(signal.ITIMER_REAL, 0) + signal.signal(signal.SIGALRM, old_handler) + + +def _pinned_model_info(model, *, opener=None): + """Bounded Hub metadata projection; no SDK full-buffer response or token.""" + ref, revision = model["ref"], model["revision"] + if (type(ref) is not str or type(revision) is not str + or any(part in {"", ".", ".."} for part in ref.split("/")) + or not revision.isascii() or len(revision) != 40 + or any(ch not in "0123456789abcdef" for ch in revision)): + raise ValueError + url = "https://huggingface.co/api/models/" + "/".join(quote(part, safe="") for part in ref.split("/")) + url += "/revision/" + revision + "?blobs=true" + raw = _bounded_hub_bytes(url, maximum=_METADATA_MAX, + deadline=time.monotonic() + 30, opener=opener) + def unique(pairs): + result = {} + for key, value in pairs: + if key in result: + raise ValueError + result[key] = value + return result + import json + value = json.loads(raw, object_pairs_hook=unique, + parse_constant=lambda value: (_ for _ in ()).throw(ValueError())) + if type(value) is not dict or type(value.get("siblings")) is not list: + raise ValueError + siblings = [] + for item in value["siblings"]: + if type(item) is not dict: + raise ValueError + lfs = item.get("lfs") + if lfs is not None and type(lfs) is not dict: + raise ValueError + siblings.append(SimpleNamespace(rfilename=item.get("rfilename"), size=item.get("size"), + blob_id=item.get("blobId"), + lfs=SimpleNamespace(sha256=lfs.get("sha256")) if lfs is not None else None)) + return SimpleNamespace(sha=value.get("sha"), siblings=siblings) + + +def prepare_pinned_model_manifest(ref, revision, *, opener=None): + """Independent upstream inventory; Git bytes are bounded and LFS weights stay remote.""" + model = {"ref": ref, "revision": revision} + members = authoritative_model_members(_pinned_model_info(model, opener=opener), + model_ref=ref, revision=revision) + if len(members) > 64 or sum(size for size, _, _ in members.values()) > 2 * 1024**3: + raise ValueError + deadline = time.monotonic() + 60 + git_total = 0 + result = [] + for path, (size, kind, upstream) in sorted(members.items()): + if kind == "lfs": + content_sha = upstream + else: + git_total += size + if git_total > _GIT_FILES_MAX: + raise ValueError + url = "https://huggingface.co/" + "/".join(quote(part, safe="") for part in ref.split("/")) + url += "/resolve/" + revision + "/" + "/".join(quote(part, safe="") for part in path.split("/")) + raw = _bounded_hub_bytes(url, maximum=max(1, size), deadline=deadline, opener=opener) + if (len(raw) != size or hashlib.sha1(f"blob {size}\0".encode("ascii") + raw).hexdigest() + != upstream): + raise ValueError + content_sha = hashlib.sha256(raw).hexdigest() + result.append({"path": path, "size_bytes": size, "kind": kind, + "upstream_digest": upstream, "sha256": content_sha}) + prepared = {"ref": ref, "revision": revision, "tokenizer_revision": revision, "members": result} + validate_model_inventory(prepared) + return prepared + + +def _source_pair(engine_repository, host_repository): + engine = inspect_git_source(engine_repository) + host = inspect_git_source(host_repository) + if (engine.dirty or not engine.pushed or host.dirty or not host.pushed + or engine.commit is None or host.commit is None): + raise ValueError + line = _git(host_repository, "ls-tree", "HEAD", "--", "synaptic-tuner") + if line != f"160000 commit {engine.commit}\tsynaptic-tuner": + raise ValueError + return engine.commit, host.commit + + +def _verify_wheel_sources(wheel, repository): + """Compare the complete fixed GRPO child source closure to wheel bytes.""" + offline = load_packaged_offline_grpo_worker_manifest() + outer = load_packaged_grpo_worker_closure() + members = {item.path.as_posix() for item in offline.closure.members} + members.update(item["path"] for item in outer.members) + with ZipFile(BytesIO(wheel)) as archive: + names = set(archive.namelist()) + if not members <= names: + raise ValueError + for name in sorted(members): + raw = stable_read(repository / name, 32 * 1024 * 1024) + if archive.read(name) != raw: + raise ValueError + return outer.digest + + +def admit_modal_grpo_constructor(*, runtime, snapshot_runtime, + engine_repository: Path, host_repository: Path, + wheel_path: Path, recipe_path: Path, + plan_ready_path: Path, dataset_path: Path, + profiles_root: Path, + input_bytes: bytes, model: dict, + library_sources: dict, attempt_ref: str): + """Exact selection with one pinned public metadata read; no model download. + + The expected SHA256s are an independently reviewed manifest; the SDK + metadata and Sandbox both revalidate upstream Git/LFS identities. + """ + try: + if (type(runtime) is not ModalHostRuntimeV1 or type(snapshot_runtime) is not ModalHostRuntimeV1 + or runtime.facts.schema_version != "synaptic-modal-packaged-runtime-facts/v2" + or snapshot_runtime.facts.schema_version != "synaptic-modal-packaged-runtime-facts/v2"): + raise ValueError + engine_commit, host_commit = _source_pair(engine_repository, host_repository) + closure = load_packaged_grpo_worker_closure() + release, deployment = runtime.release, runtime.deployment_facts + if (release.worker_closure_digest != closure.digest + or release.manifest_digest != deployment.runtime_release_digest + or runtime.facts.image_id != deployment.image_id + or runtime.facts.package_digest != release.package_digest + or runtime.facts.worker_closure_digest != closure.digest): + raise ValueError + _validate_deployment_release(deployment, release) + _validate_deployment_release(snapshot_runtime.deployment_facts, snapshot_runtime.release) + if (runtime.facts.account_ref != "synaptic-labs" + or runtime.facts.workspace_ref != "synaptic-labs" + or runtime.facts.environment_ref != "synaptic-smoke-v1" + or deployment.client_binding.environment_ref != runtime.facts.environment_ref + or snapshot_runtime.deployment_facts.client_binding != deployment.client_binding + or snapshot_runtime.facts.account_ref != runtime.facts.account_ref + or snapshot_runtime.facts.workspace_ref != runtime.facts.workspace_ref + or snapshot_runtime.facts.environment_ref != runtime.facts.environment_ref): + raise ValueError + if (runtime.quote.resource.accelerator != "L40S" + or runtime.quote.resource.accelerator_count != 1 + or runtime.quote.resource.timeout_seconds != 120 + or runtime.quote.maximum_cost_minor_units > 100 + or runtime.quote.gpu_only_timeout_estimate_minor_units > 100): + raise ValueError + wheel = stable_read(wheel_path, 256 * 1024 * 1024) + wheel_sha = hashlib.sha256(wheel).hexdigest() + if wheel_sha != release.package_digest: + raise ValueError + if _verify_wheel_sources(wheel, engine_repository) != closure.digest: + raise ValueError + expected = validate_model_inventory(model) + upstream = authoritative_model_members(_pinned_model_info(model), model_ref=model["ref"], + revision=model["revision"]) + if (len(upstream) != len(expected) + or tuple(sorted(upstream)) != tuple(item[0] for item in expected) + or any(upstream[item["path"]] != + (item["size_bytes"], item["kind"], item["upstream_digest"]) + for item in model["members"])): + raise ValueError + from tuner.cloud.grpo_constructor_diagnostic import verify_probe_input + from tuner.cloud.grpo_constructor_diagnostic import derive_probe_input + if input_bytes != derive_probe_input(recipe_path=recipe_path, + plan_ready_path=plan_ready_path, dataset_path=dataset_path, + profiles_root=profiles_root): + raise ValueError + probe = verify_probe_input(input_bytes) + workload_model = probe["workload"]["model"] + from tuner.runtime_profiles import load_runtime_profile + profile = load_runtime_profile(probe["runtime_profile"], profiles_root) + if (workload_model["model_id"] != model["ref"] + or workload_model["revision"] != model["revision"] + or workload_model["tokenizer_revision"] != model["revision"] + or profile.profile_sha256 != probe["profile_sha256"] + or profile.image != release.image_ref): + raise ValueError + from tuner.runtime.packaged_grpo_constructor_child import _SOURCES + if type(library_sources) is not dict or set(library_sources) != set(_SOURCES): + raise ValueError + dispatch = { + "schema_version": DISPATCH_SCHEMA, "attempt_ref": attempt_ref, + "source_commit": engine_commit, "host_source_commit": host_commit, + "engine_gitlink": engine_commit, "release": release.to_dict(), + "deployment_facts_digest": deployment.facts_digest, + "image_id": deployment.image_id, + "input_base64": base64.b64encode(input_bytes).decode("ascii"), + "input_sha256": hashlib.sha256(input_bytes).hexdigest(), + "model": model, "library_sources": library_sources, + } + raw = canonical(dispatch) + validate_dispatch(raw) + volume_id = _model_role(snapshot_runtime) + volume_name = dict(snapshot_runtime.volume_names_by_id).get(volume_id) + if type(volume_name) is not str: + raise ValueError + return ModalGRPOConstructorAdmission(raw, attempt_ref, engine_commit, host_commit, + wheel_sha, + release.manifest_digest, deployment.facts_digest, deployment.image_id, + snapshot_runtime.deployment_facts.facts_digest, volume_id, volume_name, + runtime.quote.quote_digest, _SEAL) + except Exception: + raise ModalGRPOConstructorUnavailable("ADMISSION") from None + + +def _read_closed_result_file(sandbox): + """Modal 1.5.4 FileIO.read(n) requests at most n bytes, without log buffering.""" + from tuner.runtime.packaged_grpo_constructor_child import _RESULT_PATH + stream = sandbox.open(_RESULT_PATH, "rb") + try: + raw = stream.read(_OUTPUT_MAX + 1) + if type(raw) is not bytes or not 0 < len(raw) <= _OUTPUT_MAX: + raise ValueError + validate_result(raw) + return raw + finally: + stream.close() + + +def _result_ready(sandbox): + from tuner.runtime.packaged_grpo_constructor_child import _READY_PATH + try: + stream = sandbox.open(_READY_PATH, "rb") + try: + return stream.read(2) == b"1" + finally: + stream.close() + except Exception: + return False + + +def _ack_result(sandbox): + from tuner.runtime.packaged_grpo_constructor_child import _ACK_PATH + stream = sandbox.open(_ACK_PATH, "xb") + try: + stream.write(b"1") + finally: + stream.close() + + +def _terminal_by_id(sdk, client, sandbox_id, *, deadline): + def read(): + handle = sdk.Sandbox.from_id(sandbox_id, client=client) + if getattr(handle, "object_id", None) != sandbox_id: + raise ValueError + return handle.poll() + return _bounded_call(read, deadline=deadline, timeout_code="constructor_terminal_unavailable") + + +def _cleanup_owned(sdk, client, sandbox, *, expected_id, deadline): + """A terminate response is never proof; fresh-ID terminal inspection is.""" + if sandbox is None or getattr(sandbox, "object_id", None) != expected_id: + return "INDETERMINATE" + try: + state = _terminal_by_id(sdk, client, expected_id, deadline=deadline) + if state is None: + _bounded_call(lambda: sandbox.terminate(wait=False), deadline=deadline, + timeout_code="constructor_cleanup_unavailable") + while time.monotonic() < deadline: + state = _terminal_by_id(sdk, client, expected_id, deadline=deadline) + if type(state) is int: + return "TERMINAL_CONFIRMED" + time.sleep(0.05) + except BaseException: + pass + return "INDETERMINATE" + + +def _catalog(storage, name): + return storage.catalog(name, encode=lambda value: canonical_bytes(value), + decode=lambda raw: parse_canonical_object(raw, name=name)) + + +def _receipt(runtime, admission, *, sandbox_id, primary, cleanup, result, terminal, + failure_phase=None): + if (primary not in {"COMPLETED", "REJECTED", "INDETERMINATE"} + or cleanup not in {"NOT_STARTED", "TERMINAL_CONFIRMED", "INDETERMINATE"} + or failure_phase not in {None, "QUOTE", "APP", "LAYOUT", "VOLUME", "IMAGE", + "CREATE", "INPUT", "OUTPUT", "TERMINAL"} + or (primary == "COMPLETED") != (failure_phase is None) + or (primary == "COMPLETED" and (result is None or terminal != 0))): + raise ValueError + body = {"schema_version": _RECEIPT, "diagnostic_type": "GRPO_CONSTRUCTOR_ONLY", + "attempt_ref": admission.attempt_ref, "dispatch_sha256": hashlib.sha256(admission.dispatch_bytes).hexdigest(), + "source_commit": admission.source_commit, + "host_source_commit": admission.host_source_commit, + "wheel_sha256": admission.wheel_sha256, + "runtime_release_digest": admission.runtime_release_digest, + "deployment_facts_digest": admission.deployment_facts_digest, + "image_id": admission.image_id, + "snapshot_deployment_facts_digest": admission.snapshot_deployment_facts_digest, + "model_cache_volume_id": admission.model_cache_volume_id, + "quote_digest": admission.quote_digest, "sandbox_id": sandbox_id, + "primary": primary, "cleanup": cleanup, "terminal": terminal, + "failure_phase": failure_phase, + "result_sha256": hashlib.sha256(result).hexdigest() if result is not None else None, + "result": validate_result(result) if result is not None else None, + "cpu_qualified": False, "training_qualified": False, "training_executed": False} + auth = ModalRuntimeQualificationHmacAuthenticator(runtime.qualification_key) + tag = auth.sign(_PURPOSE, canonical_bytes(body), QUALIFICATION_HMAC_KEY_REF) + return {"body": body, "tag_hex": tag.hex()} + + +def read_modal_grpo_constructor_receipt(runtime, storage, attempt_ref): + """Fresh journal readback; this receipt cannot satisfy CPU qualification.""" + value = _catalog(storage, "modal-grpo-constructor-results").resolve(attempt_ref) + if type(value) is not dict or set(value) != {"body", "tag_hex"}: + raise ModalGRPOConstructorUnavailable("RECEIPT") + body, tag_hex = value["body"], value["tag_hex"] + try: + tag = bytes.fromhex(tag_hex) + auth = ModalRuntimeQualificationHmacAuthenticator(runtime.qualification_key) + if (type(body) is not dict or body.get("schema_version") != _RECEIPT + or body.get("diagnostic_type") != "GRPO_CONSTRUCTOR_ONLY" + or body.get("attempt_ref") != attempt_ref + or body.get("cpu_qualified") is not False + or body.get("training_qualified") is not False + or body.get("training_executed") is not False + or body.get("primary") not in {"COMPLETED", "REJECTED", "INDETERMINATE"} + or body.get("cleanup") not in {"NOT_STARTED", "TERMINAL_CONFIRMED", "INDETERMINATE"} + or body.get("runtime_release_digest") != runtime.release.manifest_digest + or body.get("deployment_facts_digest") != runtime.deployment_facts.facts_digest + or body.get("image_id") != runtime.deployment_facts.image_id + or not auth.verify(_PURPOSE, canonical_bytes(body), tag, QUALIFICATION_HMAC_KEY_REF)): + raise ValueError + result = body.get("result") + if (result is None) != (body.get("result_sha256") is None): + raise ValueError + if result is not None: + raw = canonical(result) + validate_result(raw) + if hashlib.sha256(raw).hexdigest() != body["result_sha256"]: + raise ValueError + return value + except Exception: + raise ModalGRPOConstructorUnavailable("RECEIPT") from None + + +def execute_modal_grpo_constructor_once(*, sdk, client, client_binding, runtime, + snapshot_runtime, + private_storage, admission, + engine_repository: Path, host_repository: Path, + wheel_path: Path, recipe_path: Path, + plan_ready_path: Path, dataset_path: Path, + profiles_root: Path): + """One claimed Sandbox. No retry, training Function or CPU self-check.""" + phase = "ADMISSION" + if (type(admission) is not ModalGRPOConstructorAdmission or admission._seal is not _SEAL + or type(runtime) is not ModalHostRuntimeV1 + or type(snapshot_runtime) is not ModalHostRuntimeV1 + or client_binding != runtime.deployment_facts.client_binding + or getattr(sdk, "__version__", None) != "1.5.4"): + raise ModalGRPOConstructorUnavailable(phase) + # Revalidate source and wheel before granting a durable one-shot effect. + try: + engine_commit, host_commit = _source_pair(engine_repository, host_repository) + except Exception: + raise ModalGRPOConstructorUnavailable(phase) from None + if (engine_commit != admission.source_commit + or host_commit != admission.host_source_commit + or hashlib.sha256(stable_read(wheel_path, 256 * 1024 * 1024)).hexdigest() != admission.wheel_sha256 + or _verify_wheel_sources(stable_read(wheel_path, 256 * 1024 * 1024), engine_repository) + != runtime.release.worker_closure_digest + or runtime.release.manifest_digest != admission.runtime_release_digest + or runtime.deployment_facts.facts_digest != admission.deployment_facts_digest + or runtime.deployment_facts.image_id != admission.image_id + or snapshot_runtime.deployment_facts.facts_digest != admission.snapshot_deployment_facts_digest + or snapshot_runtime.deployment_facts.client_binding != client_binding + or _model_role(snapshot_runtime) != admission.model_cache_volume_id + or dict(snapshot_runtime.volume_names_by_id).get(admission.model_cache_volume_id) + != admission.model_cache_volume_name): + raise ModalGRPOConstructorUnavailable(phase) + try: + _, input_bytes = validate_dispatch(admission.dispatch_bytes) + from tuner.cloud.grpo_constructor_diagnostic import derive_probe_input + if input_bytes != derive_probe_input(recipe_path=recipe_path, + plan_ready_path=plan_ready_path, dataset_path=dataset_path, + profiles_root=profiles_root): + raise ValueError + except Exception: + raise ModalGRPOConstructorUnavailable(phase) from None + deadline = time.monotonic() + 240 # Begins before the first provider effect. + claim = canonical_bytes({"schema_version": _CLAIM, "attempt_ref": admission.attempt_ref, + "dispatch_sha256": hashlib.sha256(admission.dispatch_bytes).hexdigest(), + "deployment_facts_digest": admission.deployment_facts_digest, + "image_id": admission.image_id, "volume_id": admission.model_cache_volume_id, + "snapshot_deployment_facts_digest": admission.snapshot_deployment_facts_digest, + "source_commit": admission.source_commit, "wheel_sha256": admission.wheel_sha256, + "host_source_commit": admission.host_source_commit, + "quote_digest": admission.quote_digest}) + phase = "CLAIM" + private_storage.attempts.claim(admission.attempt_ref, claim) + if private_storage.attempts.resolve(admission.attempt_ref) != claim: + raise ModalGRPOConstructorUnavailable(phase) + sandbox, sandbox_id, result, terminal = None, None, None, None + primary, cleanup = "INDETERMINATE", "NOT_STARTED" + failure_phase = None + ownership = _OperationOwnership() + try: + phase = "QUOTE" + live_quote = _bounded_call(lambda: quote_modal_runtime_for_host( + sdk=sdk, client=client, client_binding=client_binding, + recipe_resource=runtime.quote.resource, + maximum_cost_minor_units=runtime.quote.maximum_cost_minor_units), + deadline=deadline, timeout_code="constructor_quote_unavailable") + if live_quote.quote_digest != admission.quote_digest: + raise ValueError + phase = "APP" + app = _bounded_call(lambda: sdk.App.lookup(runtime.facts.app_name, create_if_missing=False, + client=client, environment_name=runtime.facts.environment_ref), + deadline=deadline, timeout_code="constructor_app_unavailable") + phase = "LAYOUT" + observer = _CurrentQualificationObserver(sdk=sdk, client=client, + binding=client_binding, runtime=runtime) + _bounded_call(lambda: _observe_current_layout(observer, runtime.deployment_facts), + deadline=deadline, timeout_code="constructor_layout_unavailable") + # The old deployment is retained custody, not a current app generation. + # A new release of the same app necessarily supersedes that generation. + # Authenticate its retained facts above and rehydrate only its exact + # model-cache Volume identity below. + phase = "VOLUME" + name = admission.model_cache_volume_name + if type(name) is not str: + raise ValueError + volume = _bounded_call(lambda: sdk.Volume.from_name(name, + environment_name=runtime.facts.environment_ref, create_if_missing=False, + version=MODAL_VOLUME_V1, client=client), deadline=deadline, + timeout_code="constructor_volume_unavailable") + _bounded_call(lambda: volume.hydrate(client), deadline=deadline, + timeout_code="constructor_volume_unavailable") + if getattr(volume, "object_id", None) != admission.model_cache_volume_id: + raise ValueError + readonly = volume.with_mount_options(read_only=True) + if (getattr(readonly, "object_id", None) != admission.model_cache_volume_id + or getattr(readonly, "_read_only", None) is not True): + raise ValueError + phase = "IMAGE" + image = _bounded_call(lambda: sdk.Image.from_id(admission.image_id, client=client), + deadline=deadline, timeout_code="constructor_image_unavailable") + if getattr(image, "object_id", None) != admission.image_id: + raise ValueError + phase = "CREATE" + cleanup = "INDETERMINATE" + def late_cleanup(handle): + identity = getattr(handle, "object_id", None) + if type(identity) is str: + _cleanup_owned(sdk, client, handle, expected_id=identity, + deadline=time.monotonic() + 30) + sandbox = _bounded_call(lambda: sdk.Sandbox.create( + runtime.release.python_executable, "-I", "-B", "-m", + "tuner.runtime.packaged_grpo_constructor_child", "--supervise", + app=app, name=admission.attempt_ref, image=image, + gpu="L40S", cpu=2.0, memory=(8192, 8192), timeout=120, + idle_timeout=120, block_network=True, secrets=[], + volumes={"/mnt/synaptic-model-cache": readonly}, client=client), + deadline=deadline, timeout_code="constructor_create_indeterminate", + late_result=late_cleanup, ownership=ownership) + ownership.transfer(sandbox) + sandbox_id = getattr(sandbox, "object_id", None) + if type(sandbox_id) is not str or not sandbox_id.startswith("sb-"): + raise ValueError + call_catalog = _catalog(private_storage, "modal-grpo-constructor-sandboxes") + if (not call_catalog.publish_if_absent(admission.attempt_ref, + {"sandbox_id": sandbox_id, "dispatch_sha256": hashlib.sha256(admission.dispatch_bytes).hexdigest()}) + or call_catalog.resolve(admission.attempt_ref)["sandbox_id"] != sandbox_id): + raise ValueError + phase = "INPUT" + _bounded_call(lambda: (sandbox.stdin.write(admission.dispatch_bytes), + sandbox.stdin.write_eof(), sandbox.stdin.drain()), + deadline=deadline, timeout_code="constructor_input_indeterminate") + phase = "OUTPUT" + ready = False + while not ready and time.monotonic() < deadline: + ready = _bounded_call(lambda: _result_ready(sandbox), deadline=deadline, + timeout_code="constructor_output_unavailable") + if not ready: + terminal = _terminal_by_id(sdk, client, sandbox_id, deadline=deadline) + if terminal is not None: + raise ValueError + time.sleep(0.05) + result = _bounded_call(lambda: _read_closed_result_file(sandbox), + deadline=deadline, timeout_code="constructor_output_unavailable") + _bounded_call(lambda: _ack_result(sandbox), deadline=deadline, + timeout_code="constructor_output_unavailable") + phase = "TERMINAL" + while terminal is None and time.monotonic() < deadline: + terminal = _terminal_by_id(sdk, client, sandbox_id, deadline=deadline) + if terminal is None: + time.sleep(0.05) + if terminal != 0: + raise ValueError + primary = "COMPLETED" + except BaseException: + failure_phase = phase + primary = "INDETERMINATE" if phase in {"CREATE", "INPUT", "OUTPUT", "TERMINAL"} else "REJECTED" + finally: + if sandbox is not None and sandbox_id is not None: + cleanup = _cleanup_owned(sdk, client, sandbox, expected_id=sandbox_id, + deadline=time.monotonic() + 30) + receipt = _receipt(runtime, admission, sandbox_id=sandbox_id, + primary=primary, cleanup=cleanup, result=result, terminal=terminal, + failure_phase=failure_phase) + phase = "RECEIPT" + catalog = _catalog(private_storage, "modal-grpo-constructor-results") + if (not catalog.publish_if_absent(admission.attempt_ref, receipt) + or catalog.resolve(admission.attempt_ref) != receipt): + raise ModalGRPOConstructorUnavailable(phase) + return read_modal_grpo_constructor_receipt(runtime, private_storage, admission.attempt_ref) diff --git a/tuner/training/modal_standalone_runner.py b/tuner/training/modal_standalone_runner.py index fb7be733..124b9a1c 100644 --- a/tuner/training/modal_standalone_runner.py +++ b/tuner/training/modal_standalone_runner.py @@ -254,7 +254,7 @@ def _private_root() -> Path: return training_state_root() -def prepare_modal_standalone_local(*, plan, context): +def prepare_modal_standalone_local(*, plan, context, state_root: Path | None = None): """The same mandatory local-only preparation used by normal execution.""" validate_local_launcher() from tuner.execution.providers.modal.runtime_build import current_clean_source_commit @@ -271,7 +271,9 @@ def prepare_modal_standalone_local(*, plan, context): if (admitted.profile_sha256 != plan.profile_sha256 or admitted.image != plan.profile_base_image or intent["intent_digest"] != plan.runtime_material_intent_digest): raise ModalLocalPreparationUnavailable("PROFILE_BINDING") - return prepare_local_inputs(profile_path, root=_private_root(), expected_intent=plan.runtime_material_intent_digest, source_root=context.engine_root) + return prepare_local_inputs(profile_path, root=_private_root() if state_root is None else state_root, + expected_intent=plan.runtime_material_intent_digest, + source_root=context.engine_root) def _attempt_journal(root: Path, attempt_ref: str, *, fresh_attempt: bool) -> Path: @@ -333,7 +335,8 @@ def _qualified_runtime(*, modal: object, plan: ModalSFTRecipePlanV1 | ModalGRPOR context: object, resources: ResourceSpec, storage: ModalHostStorageV1, effect_id: str, modal_profile: str, modal_environment: str, - builder_cache_root: Path, metrics_admission=None, local_inputs=None): + builder_cache_root: Path, metrics_admission=None, local_inputs=None, + hf_token: str | None = None): if type(plan) not in (ModalSFTRecipePlanV1, ModalGRPORecipePlanV1): raise ModalStandaloneRunUnavailable("modal_host_configuration_invalid") grpo = type(plan) is ModalGRPORecipePlanV1 @@ -361,6 +364,7 @@ def _qualified_runtime(*, modal: object, plan: ModalSFTRecipePlanV1 | ModalGRPOR secrets_port = _HostSecrets({ "SYNAPTIC_TRAIN_AUTHORITY_KEY": secrets.token_hex(32), "SYNAPTIC_TRAIN_READER_KEY": secrets.token_hex(32), + **({"HF_TOKEN": hf_token} if hf_token is not None else {}), }) hf_ref = SecretRef("env", "HF_TOKEN") secrets_port.resolve(hf_ref)