diff --git a/.agents/skills/fine-tuning/SKILL.md b/.agents/skills/fine-tuning/SKILL.md index 73c91863..7aae0806 100644 --- a/.agents/skills/fine-tuning/SKILL.md +++ b/.agents/skills/fine-tuning/SKILL.md @@ -57,7 +57,8 @@ Train language models with SFT, KTO, and GRPO locally or on supported cloud prov | Check SFT loss masking (tokenizer only) | `python tuner.py doctor sft-mask --dataset-path --model ` | | Inspect native training host candidate | `python tuner.py doctor native-training --json` — read-only platform/Python/package metadata; see `reference/native-local-training.md` | | Diagnose an offline SFT worker rejection | Read the exact retained `trainer.stderr.log`; closed phase/category/milestone labels locate a boundary, not a cause or retry authority; see `reference/sft-training.md` | -| Native MLX prepared SFT candidate | Internal exact-token batch/loss and runtime comparison only; no native launch CLI or API provider yet; see `reference/native-local-training.md` | +| Native MLX prepared SFT candidate | Gated `ProviderFamilyV1` through existing TrainingAPI/RunsAPI; isolated worker, signed host catalog and native artifacts; no hardware qualification or launch CLI yet; see `reference/native-local-training.md` | +| Capture reviewed native runtime bytes | `python -B scripts/capture_native_mlx_runtime.py --config --dependency-lock --output ` — observation only; explicit roots, dependencies and original pins required; see `reference/native-local-training.md` | | Plan a derived training image | `python scripts/qualify_derived_training_image.py plan --config Trainers/image_profiles/.yaml` | | Warm Space scaffold | `python3 Trainers/cloud/scripts/manage_space.py render --template vllm_warm --output-dir /tmp/my-space --base-image ghcr.io//:` | | Warm Space deploy | `python3 Trainers/cloud/scripts/manage_space.py deploy --space-id / --template vllm_warm --base-image ghcr.io//: --hardware a10g-small --sleep-time 3600 --var BASE_MODEL=` | diff --git a/.agents/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml b/.agents/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml index ca01872f..02f5e50a 100644 --- a/.agents/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml +++ b/.agents/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml @@ -8,7 +8,7 @@ model: runtime: provider: modal lock_path: tuner/execution/providers/modal/modal-runtime-v1.lock.json - expected_lock_sha256: 4f3fa81c9aeb9130258dec23f2b841e6dbdf2d686d1cf6b50249e743c3d496ca + expected_lock_sha256: 1f7422b56bf4d1c7bd07dfc440da8263719d7e74dc4ceeeb94100f9680a48087 input: jsonl_path: /replace/with/private/prepared-dataset.jsonl mode: text diff --git a/.agents/skills/fine-tuning/reference/native-local-training.md b/.agents/skills/fine-tuning/reference/native-local-training.md index 3831ab8f..f7684383 100644 --- a/.agents/skills/fine-tuning/reference/native-local-training.md +++ b/.agents/skills/fine-tuning/reference/native-local-training.md @@ -45,11 +45,11 @@ remain ignored. Training shuffles deterministically within its own split; validation retains every row in declared order, including partial batches. The custom hooks target pinned MLX-LM 0.32.2. One batch is one optimizer update; -gradient accumulation and checkpointing are deliberately unavailable in this +gradient accumulation and activation checkpointing are deliberately unavailable in this increment. Upstream accumulation averages microbatch means, which needs a separate objective review for unequal supervised-token counts. Model, LoRA, -optimizer and learning-rate policy must be prepared explicitly by the future -child composition; this worker does not infer or silently apply those controls. +optimizer and learning-rate policy are prepared explicitly by the isolated +child composition; this worker does not infer those controls. `tuner/runtime/native_mlx_runtime.py` compares host-supplied exact interpreter and complete distribution pins against the running candidate before worker ML @@ -62,5 +62,130 @@ into the shared interpreter or treat a newly observed inventory as approved. There is no native launch CLI yet. The internal worker may run only in an independently admitted, host-owned bounded child. The desktop plan lists the -missing native runtime material, child lifecycle, artifact roles and API joins. +implemented runtime/family boundaries and remaining consumer/qualification work. Do not call the worker as an ad hoc substitute for that composition. + +## Execution-boundary continuation + +The internal runtime modules now include a read-only native material capture, +retained-source/tokenizer join, explicit LoRA/constant-AdamW composition, +independent safetensors delta verifier, and owned subprocess guardian. These are +reviewable runtime boundaries. The gated family described below composes them +through the existing public coordinator; no global provider registration or +native launch CLI is supplied. + +`scripts/capture_native_mlx_runtime.py` takes a JSON object with `capture` and +`expectation`. `expectation` uses the existing `MLXRuntimeExpectation` fields. +`capture` contains absolute `stdlib`/`packages` roots, sorted `sources` trees, +sorted explicit external `files`, sorted `system_libraries`, and sorted +`aliases` pairs of absolute alias path and exact symlink spelling. Every tree +file, including startup `.pth` and bytecode, is captured. Unlisted aliases reject. +Homebrew's framework aliases must resolve to explicit captured bytes. Its +stdlib `site-packages` alias is recorded but excluded: the isolated child uses +`-I -S -B` and explicitly binds only its captured package search path. + +Run the selected interpreter with `-B`, from an independently reviewed source +checkout, to avoid creating bytecode while capturing. Output requires an +existing owner-private parent and is exclusive; no environment repair or +installation occurs. `--check --expected-digest ` +revalidates the exact original configuration and capture. Unknown/ambiguous +Mach-O dependencies, uncaptured external files and extra declared OS libraries +reject. OS shared-cache dependencies are explicitly trusted through the recorded +OS/kernel build, not falsely described as hashed ordinary files. + +This captures installed bytes and static native dependencies. It does **not** +prove selected wheel archive provenance, attest dynamic `dlopen` behavior, +freeze the filesystem against a malicious same-uid process, or independently +grant execution. Root review must reconcile the original hash-required install +evidence and selected archives; public capability remains gated. + +`native_mlx_input.py` consumes the existing retained stream lease, verifies +identity/hash/size, and feeds the established JSONL publication rows to shared +`materialize_sft_example`. It retains row IDs and splits without filtering or +repair. The tokenizer loads only from a verified private local snapshot with +`trust_remote_code=False` and `local_files_only=True`. Model and tokenizer +snapshots, exact source, rendering arguments and worker controls are committed +in the bound input. The host still owns publication semantic/lineage validation +and binding to an actual compiled workload. + +`native_mlx_composition.py` loads a configured local MLX snapshot, selects every +LoRA target/layer explicitly, checks the realized trainable names/shapes and uses +explicit AdamW parameters with constant schedule. Unsupported schedules reject. +It checks the optimizer's actual step count. The native adapter verifier checks +tensor headers, bounds, names/shapes/dtypes, finite values and a nonzero numeric +parameter delta. Final validation is a distinct pass **after reloading the saved +adapter**; upstream in-loop validation occurs before the last optimizer update. +It saves native `adapter_config.json`, `input_binding.json` and +`native_result.json` exclusively. These worker files become authenticated public +artifact roles only through the family verifier described below. + +`native_process.py` records claim/spawn/outcome through the existing +`DurableRecordStorePort` in the effects partition. There is no engine database. +The stdlib guardian owns the worker group, drains bounded output without +exposing its contents, enforces time/file bounds, and terminates on owner-pipe +loss or cancellation. Darwin uses kqueue process notifications because the +qualification CPython does not expose `os.waitid`. Linux tests use waitid without +reaping. Persisted PIDs are diagnostic only; another process cannot acquire +signal authority by reading them. Interrupted submission/publication retains +uncertainty and never auto-replays. This is trusted-worker supervision, not an +OS sandbox; escaped hostile descendants and guardian failure remain outside the +current evidence. Real Darwin lifecycle tests remain required. + +Before ML imports/training, independently review the isolated child bootstrap, +authenticated compiled workload/native material, public provider-family +composition and native artifact roles. The bounded next proposal is +`Trainers/mlx_sft_mac/runtime-candidate/sft-qualification-proposal.json`; its +status is review-required and it is not executable admission. + +## Public coordinator continuation + +`native_mlx_compilation.py` recompiles `TrainingInputV1` plus the retained +`PreparedTrainingInputIdentity` and explicit `NativeMLXProfile` into a normal +`CompiledWorkload`. Parsing re-derives the complete document. Model/tokenizer +pins, loss semantics, chat arguments, batch/sequence/update/seed/checkpoint +controls, LoRA and constant AdamW are explicit. Accumulation, epochs, DoRA, +RSLoRA, implicit splits, packing and unsupported checkpoint-pruning policies +reject. Profiles declare the alpha/rank scale conversion; no model switch is +embedded in a launcher. + +`native_mlx_child.py` is an internal `-I -S -B` entrypoint. It restricts import +roots, checks imported source/extension origins against capture members and +compiles pinned Python source rather than ambient bytecode. It joins the exact +staged dataset, capture, lock and snapshots to the recompiled workload. It is +not a public invocation or an independent source/qualification authority. +`native_runtime_spec` represents the capture with the existing `RuntimeSpec` +`native_environment` discriminator; it invents no OCI image. Legacy OCI +canonical bytes are unchanged. + +`native_mlx_planning.py` supplies immutable planning/preparation adapters. +`native_mlx_family.py` supplies the existing `ProviderFamilyV1`: stage, bounded +synchronous submit, authenticated known-run observation and artifact streaming. +Use it with `compose_reference_host`; Syntunia's existing loader/resolver, +grant, record, stream, secret and clock ports remain authoritative. The consumer +must authenticate actual source evidence and current hardware before signing +command material. `NativeCommandCatalogV1` retains create-only signed material +through the existing record port. Signed terminal evidence must also match the +retained process outcome on every read. It is not an engine database or replay grant. + +No accepting hardware authority is shipped. Candidate descriptors have +observe/artifact capabilities disabled and preflight is not ready. A +`NativeQualificationV1` contains exact selected profile/source/runtime/hardware +and two distinct retained attempt-evidence references; its independent host +authority must authenticate the supporting real runs, selected wheel/install +provenance and final-adapter/public-readback evidence. Reference hashes alone +are not qualification. Public logs, cancellation and reconciliation remain +disabled even with this narrow qualification. The guardian's internal stop +path does not promote public cancellation. + +`native_mlx_publication.py` independently checks the exact output inventory, +workload/input binding, optimizer count, finite saved tensor delta, retained +checkpoint tensors and final-adapter validation identity. It emits the five +existing roles. `final_model` is a deterministic POSIX tar containing the MLX +adapter/config/checkpoints and pinned base manifest; `tokenizer` is an explicit +configured member archive from the verified snapshot. Lineage/metrics retain +the exact worker bindings. Finite file/aggregate budgets and public 1 MiB +stream chunks apply. CPU verification does not rerun the final evaluation. + +See `docs/plans/native-desktop-training-api-v1.md` for remaining host assembly, +review and real-Mac acceptance. The CPU facade tests substitute the ML child; +they are not a hardware result and must never unlock the candidate. diff --git a/.claude/skills/fine-tuning/SKILL.md b/.claude/skills/fine-tuning/SKILL.md index 73c91863..7aae0806 100644 --- a/.claude/skills/fine-tuning/SKILL.md +++ b/.claude/skills/fine-tuning/SKILL.md @@ -57,7 +57,8 @@ Train language models with SFT, KTO, and GRPO locally or on supported cloud prov | Check SFT loss masking (tokenizer only) | `python tuner.py doctor sft-mask --dataset-path --model ` | | Inspect native training host candidate | `python tuner.py doctor native-training --json` — read-only platform/Python/package metadata; see `reference/native-local-training.md` | | Diagnose an offline SFT worker rejection | Read the exact retained `trainer.stderr.log`; closed phase/category/milestone labels locate a boundary, not a cause or retry authority; see `reference/sft-training.md` | -| Native MLX prepared SFT candidate | Internal exact-token batch/loss and runtime comparison only; no native launch CLI or API provider yet; see `reference/native-local-training.md` | +| Native MLX prepared SFT candidate | Gated `ProviderFamilyV1` through existing TrainingAPI/RunsAPI; isolated worker, signed host catalog and native artifacts; no hardware qualification or launch CLI yet; see `reference/native-local-training.md` | +| Capture reviewed native runtime bytes | `python -B scripts/capture_native_mlx_runtime.py --config --dependency-lock --output ` — observation only; explicit roots, dependencies and original pins required; see `reference/native-local-training.md` | | Plan a derived training image | `python scripts/qualify_derived_training_image.py plan --config Trainers/image_profiles/.yaml` | | Warm Space scaffold | `python3 Trainers/cloud/scripts/manage_space.py render --template vllm_warm --output-dir /tmp/my-space --base-image ghcr.io//:` | | Warm Space deploy | `python3 Trainers/cloud/scripts/manage_space.py deploy --space-id / --template vllm_warm --base-image ghcr.io//: --hardware a10g-small --sleep-time 3600 --var BASE_MODEL=` | diff --git a/.claude/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml b/.claude/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml index ca01872f..02f5e50a 100644 --- a/.claude/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml +++ b/.claude/skills/fine-tuning/configs/qwen35_4b_token_profile.yaml @@ -8,7 +8,7 @@ model: runtime: provider: modal lock_path: tuner/execution/providers/modal/modal-runtime-v1.lock.json - expected_lock_sha256: 4f3fa81c9aeb9130258dec23f2b841e6dbdf2d686d1cf6b50249e743c3d496ca + expected_lock_sha256: 1f7422b56bf4d1c7bd07dfc440da8263719d7e74dc4ceeeb94100f9680a48087 input: jsonl_path: /replace/with/private/prepared-dataset.jsonl mode: text diff --git a/.claude/skills/fine-tuning/reference/native-local-training.md b/.claude/skills/fine-tuning/reference/native-local-training.md index 3831ab8f..f7684383 100644 --- a/.claude/skills/fine-tuning/reference/native-local-training.md +++ b/.claude/skills/fine-tuning/reference/native-local-training.md @@ -45,11 +45,11 @@ remain ignored. Training shuffles deterministically within its own split; validation retains every row in declared order, including partial batches. The custom hooks target pinned MLX-LM 0.32.2. One batch is one optimizer update; -gradient accumulation and checkpointing are deliberately unavailable in this +gradient accumulation and activation checkpointing are deliberately unavailable in this increment. Upstream accumulation averages microbatch means, which needs a separate objective review for unequal supervised-token counts. Model, LoRA, -optimizer and learning-rate policy must be prepared explicitly by the future -child composition; this worker does not infer or silently apply those controls. +optimizer and learning-rate policy are prepared explicitly by the isolated +child composition; this worker does not infer those controls. `tuner/runtime/native_mlx_runtime.py` compares host-supplied exact interpreter and complete distribution pins against the running candidate before worker ML @@ -62,5 +62,130 @@ into the shared interpreter or treat a newly observed inventory as approved. There is no native launch CLI yet. The internal worker may run only in an independently admitted, host-owned bounded child. The desktop plan lists the -missing native runtime material, child lifecycle, artifact roles and API joins. +implemented runtime/family boundaries and remaining consumer/qualification work. Do not call the worker as an ad hoc substitute for that composition. + +## Execution-boundary continuation + +The internal runtime modules now include a read-only native material capture, +retained-source/tokenizer join, explicit LoRA/constant-AdamW composition, +independent safetensors delta verifier, and owned subprocess guardian. These are +reviewable runtime boundaries. The gated family described below composes them +through the existing public coordinator; no global provider registration or +native launch CLI is supplied. + +`scripts/capture_native_mlx_runtime.py` takes a JSON object with `capture` and +`expectation`. `expectation` uses the existing `MLXRuntimeExpectation` fields. +`capture` contains absolute `stdlib`/`packages` roots, sorted `sources` trees, +sorted explicit external `files`, sorted `system_libraries`, and sorted +`aliases` pairs of absolute alias path and exact symlink spelling. Every tree +file, including startup `.pth` and bytecode, is captured. Unlisted aliases reject. +Homebrew's framework aliases must resolve to explicit captured bytes. Its +stdlib `site-packages` alias is recorded but excluded: the isolated child uses +`-I -S -B` and explicitly binds only its captured package search path. + +Run the selected interpreter with `-B`, from an independently reviewed source +checkout, to avoid creating bytecode while capturing. Output requires an +existing owner-private parent and is exclusive; no environment repair or +installation occurs. `--check --expected-digest ` +revalidates the exact original configuration and capture. Unknown/ambiguous +Mach-O dependencies, uncaptured external files and extra declared OS libraries +reject. OS shared-cache dependencies are explicitly trusted through the recorded +OS/kernel build, not falsely described as hashed ordinary files. + +This captures installed bytes and static native dependencies. It does **not** +prove selected wheel archive provenance, attest dynamic `dlopen` behavior, +freeze the filesystem against a malicious same-uid process, or independently +grant execution. Root review must reconcile the original hash-required install +evidence and selected archives; public capability remains gated. + +`native_mlx_input.py` consumes the existing retained stream lease, verifies +identity/hash/size, and feeds the established JSONL publication rows to shared +`materialize_sft_example`. It retains row IDs and splits without filtering or +repair. The tokenizer loads only from a verified private local snapshot with +`trust_remote_code=False` and `local_files_only=True`. Model and tokenizer +snapshots, exact source, rendering arguments and worker controls are committed +in the bound input. The host still owns publication semantic/lineage validation +and binding to an actual compiled workload. + +`native_mlx_composition.py` loads a configured local MLX snapshot, selects every +LoRA target/layer explicitly, checks the realized trainable names/shapes and uses +explicit AdamW parameters with constant schedule. Unsupported schedules reject. +It checks the optimizer's actual step count. The native adapter verifier checks +tensor headers, bounds, names/shapes/dtypes, finite values and a nonzero numeric +parameter delta. Final validation is a distinct pass **after reloading the saved +adapter**; upstream in-loop validation occurs before the last optimizer update. +It saves native `adapter_config.json`, `input_binding.json` and +`native_result.json` exclusively. These worker files become authenticated public +artifact roles only through the family verifier described below. + +`native_process.py` records claim/spawn/outcome through the existing +`DurableRecordStorePort` in the effects partition. There is no engine database. +The stdlib guardian owns the worker group, drains bounded output without +exposing its contents, enforces time/file bounds, and terminates on owner-pipe +loss or cancellation. Darwin uses kqueue process notifications because the +qualification CPython does not expose `os.waitid`. Linux tests use waitid without +reaping. Persisted PIDs are diagnostic only; another process cannot acquire +signal authority by reading them. Interrupted submission/publication retains +uncertainty and never auto-replays. This is trusted-worker supervision, not an +OS sandbox; escaped hostile descendants and guardian failure remain outside the +current evidence. Real Darwin lifecycle tests remain required. + +Before ML imports/training, independently review the isolated child bootstrap, +authenticated compiled workload/native material, public provider-family +composition and native artifact roles. The bounded next proposal is +`Trainers/mlx_sft_mac/runtime-candidate/sft-qualification-proposal.json`; its +status is review-required and it is not executable admission. + +## Public coordinator continuation + +`native_mlx_compilation.py` recompiles `TrainingInputV1` plus the retained +`PreparedTrainingInputIdentity` and explicit `NativeMLXProfile` into a normal +`CompiledWorkload`. Parsing re-derives the complete document. Model/tokenizer +pins, loss semantics, chat arguments, batch/sequence/update/seed/checkpoint +controls, LoRA and constant AdamW are explicit. Accumulation, epochs, DoRA, +RSLoRA, implicit splits, packing and unsupported checkpoint-pruning policies +reject. Profiles declare the alpha/rank scale conversion; no model switch is +embedded in a launcher. + +`native_mlx_child.py` is an internal `-I -S -B` entrypoint. It restricts import +roots, checks imported source/extension origins against capture members and +compiles pinned Python source rather than ambient bytecode. It joins the exact +staged dataset, capture, lock and snapshots to the recompiled workload. It is +not a public invocation or an independent source/qualification authority. +`native_runtime_spec` represents the capture with the existing `RuntimeSpec` +`native_environment` discriminator; it invents no OCI image. Legacy OCI +canonical bytes are unchanged. + +`native_mlx_planning.py` supplies immutable planning/preparation adapters. +`native_mlx_family.py` supplies the existing `ProviderFamilyV1`: stage, bounded +synchronous submit, authenticated known-run observation and artifact streaming. +Use it with `compose_reference_host`; Syntunia's existing loader/resolver, +grant, record, stream, secret and clock ports remain authoritative. The consumer +must authenticate actual source evidence and current hardware before signing +command material. `NativeCommandCatalogV1` retains create-only signed material +through the existing record port. Signed terminal evidence must also match the +retained process outcome on every read. It is not an engine database or replay grant. + +No accepting hardware authority is shipped. Candidate descriptors have +observe/artifact capabilities disabled and preflight is not ready. A +`NativeQualificationV1` contains exact selected profile/source/runtime/hardware +and two distinct retained attempt-evidence references; its independent host +authority must authenticate the supporting real runs, selected wheel/install +provenance and final-adapter/public-readback evidence. Reference hashes alone +are not qualification. Public logs, cancellation and reconciliation remain +disabled even with this narrow qualification. The guardian's internal stop +path does not promote public cancellation. + +`native_mlx_publication.py` independently checks the exact output inventory, +workload/input binding, optimizer count, finite saved tensor delta, retained +checkpoint tensors and final-adapter validation identity. It emits the five +existing roles. `final_model` is a deterministic POSIX tar containing the MLX +adapter/config/checkpoints and pinned base manifest; `tokenizer` is an explicit +configured member archive from the verified snapshot. Lineage/metrics retain +the exact worker bindings. Finite file/aggregate budgets and public 1 MiB +stream chunks apply. CPU verification does not rerun the final evaluation. + +See `docs/plans/native-desktop-training-api-v1.md` for remaining host assembly, +review and real-Mac acceptance. The CPU facade tests substitute the ML child; +they are not a hardware result and must never unlock the candidate. diff --git a/.skills/fine-tuning/SKILL.md b/.skills/fine-tuning/SKILL.md index 73c91863..7aae0806 100644 --- a/.skills/fine-tuning/SKILL.md +++ b/.skills/fine-tuning/SKILL.md @@ -57,7 +57,8 @@ Train language models with SFT, KTO, and GRPO locally or on supported cloud prov | Check SFT loss masking (tokenizer only) | `python tuner.py doctor sft-mask --dataset-path --model ` | | Inspect native training host candidate | `python tuner.py doctor native-training --json` — read-only platform/Python/package metadata; see `reference/native-local-training.md` | | Diagnose an offline SFT worker rejection | Read the exact retained `trainer.stderr.log`; closed phase/category/milestone labels locate a boundary, not a cause or retry authority; see `reference/sft-training.md` | -| Native MLX prepared SFT candidate | Internal exact-token batch/loss and runtime comparison only; no native launch CLI or API provider yet; see `reference/native-local-training.md` | +| Native MLX prepared SFT candidate | Gated `ProviderFamilyV1` through existing TrainingAPI/RunsAPI; isolated worker, signed host catalog and native artifacts; no hardware qualification or launch CLI yet; see `reference/native-local-training.md` | +| Capture reviewed native runtime bytes | `python -B scripts/capture_native_mlx_runtime.py --config --dependency-lock --output ` — observation only; explicit roots, dependencies and original pins required; see `reference/native-local-training.md` | | Plan a derived training image | `python scripts/qualify_derived_training_image.py plan --config Trainers/image_profiles/.yaml` | | Warm Space scaffold | `python3 Trainers/cloud/scripts/manage_space.py render --template vllm_warm --output-dir /tmp/my-space --base-image ghcr.io//:` | | Warm Space deploy | `python3 Trainers/cloud/scripts/manage_space.py deploy --space-id / --template vllm_warm --base-image ghcr.io//: --hardware a10g-small --sleep-time 3600 --var BASE_MODEL=` | diff --git a/.skills/fine-tuning/configs/qwen35_4b_token_profile.yaml b/.skills/fine-tuning/configs/qwen35_4b_token_profile.yaml index ca01872f..02f5e50a 100644 --- a/.skills/fine-tuning/configs/qwen35_4b_token_profile.yaml +++ b/.skills/fine-tuning/configs/qwen35_4b_token_profile.yaml @@ -8,7 +8,7 @@ model: runtime: provider: modal lock_path: tuner/execution/providers/modal/modal-runtime-v1.lock.json - expected_lock_sha256: 4f3fa81c9aeb9130258dec23f2b841e6dbdf2d686d1cf6b50249e743c3d496ca + expected_lock_sha256: 1f7422b56bf4d1c7bd07dfc440da8263719d7e74dc4ceeeb94100f9680a48087 input: jsonl_path: /replace/with/private/prepared-dataset.jsonl mode: text diff --git a/.skills/fine-tuning/reference/native-local-training.md b/.skills/fine-tuning/reference/native-local-training.md index 3831ab8f..f7684383 100644 --- a/.skills/fine-tuning/reference/native-local-training.md +++ b/.skills/fine-tuning/reference/native-local-training.md @@ -45,11 +45,11 @@ remain ignored. Training shuffles deterministically within its own split; validation retains every row in declared order, including partial batches. The custom hooks target pinned MLX-LM 0.32.2. One batch is one optimizer update; -gradient accumulation and checkpointing are deliberately unavailable in this +gradient accumulation and activation checkpointing are deliberately unavailable in this increment. Upstream accumulation averages microbatch means, which needs a separate objective review for unequal supervised-token counts. Model, LoRA, -optimizer and learning-rate policy must be prepared explicitly by the future -child composition; this worker does not infer or silently apply those controls. +optimizer and learning-rate policy are prepared explicitly by the isolated +child composition; this worker does not infer those controls. `tuner/runtime/native_mlx_runtime.py` compares host-supplied exact interpreter and complete distribution pins against the running candidate before worker ML @@ -62,5 +62,130 @@ into the shared interpreter or treat a newly observed inventory as approved. There is no native launch CLI yet. The internal worker may run only in an independently admitted, host-owned bounded child. The desktop plan lists the -missing native runtime material, child lifecycle, artifact roles and API joins. +implemented runtime/family boundaries and remaining consumer/qualification work. Do not call the worker as an ad hoc substitute for that composition. + +## Execution-boundary continuation + +The internal runtime modules now include a read-only native material capture, +retained-source/tokenizer join, explicit LoRA/constant-AdamW composition, +independent safetensors delta verifier, and owned subprocess guardian. These are +reviewable runtime boundaries. The gated family described below composes them +through the existing public coordinator; no global provider registration or +native launch CLI is supplied. + +`scripts/capture_native_mlx_runtime.py` takes a JSON object with `capture` and +`expectation`. `expectation` uses the existing `MLXRuntimeExpectation` fields. +`capture` contains absolute `stdlib`/`packages` roots, sorted `sources` trees, +sorted explicit external `files`, sorted `system_libraries`, and sorted +`aliases` pairs of absolute alias path and exact symlink spelling. Every tree +file, including startup `.pth` and bytecode, is captured. Unlisted aliases reject. +Homebrew's framework aliases must resolve to explicit captured bytes. Its +stdlib `site-packages` alias is recorded but excluded: the isolated child uses +`-I -S -B` and explicitly binds only its captured package search path. + +Run the selected interpreter with `-B`, from an independently reviewed source +checkout, to avoid creating bytecode while capturing. Output requires an +existing owner-private parent and is exclusive; no environment repair or +installation occurs. `--check --expected-digest ` +revalidates the exact original configuration and capture. Unknown/ambiguous +Mach-O dependencies, uncaptured external files and extra declared OS libraries +reject. OS shared-cache dependencies are explicitly trusted through the recorded +OS/kernel build, not falsely described as hashed ordinary files. + +This captures installed bytes and static native dependencies. It does **not** +prove selected wheel archive provenance, attest dynamic `dlopen` behavior, +freeze the filesystem against a malicious same-uid process, or independently +grant execution. Root review must reconcile the original hash-required install +evidence and selected archives; public capability remains gated. + +`native_mlx_input.py` consumes the existing retained stream lease, verifies +identity/hash/size, and feeds the established JSONL publication rows to shared +`materialize_sft_example`. It retains row IDs and splits without filtering or +repair. The tokenizer loads only from a verified private local snapshot with +`trust_remote_code=False` and `local_files_only=True`. Model and tokenizer +snapshots, exact source, rendering arguments and worker controls are committed +in the bound input. The host still owns publication semantic/lineage validation +and binding to an actual compiled workload. + +`native_mlx_composition.py` loads a configured local MLX snapshot, selects every +LoRA target/layer explicitly, checks the realized trainable names/shapes and uses +explicit AdamW parameters with constant schedule. Unsupported schedules reject. +It checks the optimizer's actual step count. The native adapter verifier checks +tensor headers, bounds, names/shapes/dtypes, finite values and a nonzero numeric +parameter delta. Final validation is a distinct pass **after reloading the saved +adapter**; upstream in-loop validation occurs before the last optimizer update. +It saves native `adapter_config.json`, `input_binding.json` and +`native_result.json` exclusively. These worker files become authenticated public +artifact roles only through the family verifier described below. + +`native_process.py` records claim/spawn/outcome through the existing +`DurableRecordStorePort` in the effects partition. There is no engine database. +The stdlib guardian owns the worker group, drains bounded output without +exposing its contents, enforces time/file bounds, and terminates on owner-pipe +loss or cancellation. Darwin uses kqueue process notifications because the +qualification CPython does not expose `os.waitid`. Linux tests use waitid without +reaping. Persisted PIDs are diagnostic only; another process cannot acquire +signal authority by reading them. Interrupted submission/publication retains +uncertainty and never auto-replays. This is trusted-worker supervision, not an +OS sandbox; escaped hostile descendants and guardian failure remain outside the +current evidence. Real Darwin lifecycle tests remain required. + +Before ML imports/training, independently review the isolated child bootstrap, +authenticated compiled workload/native material, public provider-family +composition and native artifact roles. The bounded next proposal is +`Trainers/mlx_sft_mac/runtime-candidate/sft-qualification-proposal.json`; its +status is review-required and it is not executable admission. + +## Public coordinator continuation + +`native_mlx_compilation.py` recompiles `TrainingInputV1` plus the retained +`PreparedTrainingInputIdentity` and explicit `NativeMLXProfile` into a normal +`CompiledWorkload`. Parsing re-derives the complete document. Model/tokenizer +pins, loss semantics, chat arguments, batch/sequence/update/seed/checkpoint +controls, LoRA and constant AdamW are explicit. Accumulation, epochs, DoRA, +RSLoRA, implicit splits, packing and unsupported checkpoint-pruning policies +reject. Profiles declare the alpha/rank scale conversion; no model switch is +embedded in a launcher. + +`native_mlx_child.py` is an internal `-I -S -B` entrypoint. It restricts import +roots, checks imported source/extension origins against capture members and +compiles pinned Python source rather than ambient bytecode. It joins the exact +staged dataset, capture, lock and snapshots to the recompiled workload. It is +not a public invocation or an independent source/qualification authority. +`native_runtime_spec` represents the capture with the existing `RuntimeSpec` +`native_environment` discriminator; it invents no OCI image. Legacy OCI +canonical bytes are unchanged. + +`native_mlx_planning.py` supplies immutable planning/preparation adapters. +`native_mlx_family.py` supplies the existing `ProviderFamilyV1`: stage, bounded +synchronous submit, authenticated known-run observation and artifact streaming. +Use it with `compose_reference_host`; Syntunia's existing loader/resolver, +grant, record, stream, secret and clock ports remain authoritative. The consumer +must authenticate actual source evidence and current hardware before signing +command material. `NativeCommandCatalogV1` retains create-only signed material +through the existing record port. Signed terminal evidence must also match the +retained process outcome on every read. It is not an engine database or replay grant. + +No accepting hardware authority is shipped. Candidate descriptors have +observe/artifact capabilities disabled and preflight is not ready. A +`NativeQualificationV1` contains exact selected profile/source/runtime/hardware +and two distinct retained attempt-evidence references; its independent host +authority must authenticate the supporting real runs, selected wheel/install +provenance and final-adapter/public-readback evidence. Reference hashes alone +are not qualification. Public logs, cancellation and reconciliation remain +disabled even with this narrow qualification. The guardian's internal stop +path does not promote public cancellation. + +`native_mlx_publication.py` independently checks the exact output inventory, +workload/input binding, optimizer count, finite saved tensor delta, retained +checkpoint tensors and final-adapter validation identity. It emits the five +existing roles. `final_model` is a deterministic POSIX tar containing the MLX +adapter/config/checkpoints and pinned base manifest; `tokenizer` is an explicit +configured member archive from the verified snapshot. Lineage/metrics retain +the exact worker bindings. Finite file/aggregate budgets and public 1 MiB +stream chunks apply. CPU verification does not rerun the final evaluation. + +See `docs/plans/native-desktop-training-api-v1.md` for remaining host assembly, +review and real-Mac acceptance. The CPU facade tests substitute the ML child; +they are not a hardware result and must never unlock the candidate. diff --git a/AGENTS.md b/AGENTS.md index 381fcb17..ef099261 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -45,12 +45,40 @@ This repository has a few cloud-training constraints that are easy to relearn th truncation, or dropped tail batches. Reject unsupported attention layouts and failed preprocessing diagnostics before ML imports. - The internal `tuner/runtime/native_mlx_sft.py` worker and candidate metadata - comparison are not a native provider, public runtime material, or launch + comparison alone are not a native provider, public runtime material, or launch authority. A finite iteration count is not a wall-clock/process bound. Keep model/optimizer preparation, exact runtime/source admission, private paths, durable claims, child lifecycle and native artifact verification host-owned. Do not enable native training in Syntunia before those joins and real Mac qualification are complete. See `docs/plans/native-desktop-training-api-v1.md`. +- Capture native runtime files and linked libraries, including the Homebrew + Python framework, startup files and bytecode. A stub executable hash or package + metadata match cannot attest this closure. Treat the OS shared cache as an + explicit exact-build trust boundary. Capture observations are not host grants + or proof that installed bytes came from selected hash-locked wheel archives. +- Use actual Darwin filesystem/process interfaces. The qualification Mac's + CPython 3.11.14 has kqueue but no `os.waitid`; never assume Linux pidfds, /proc + or waitid availability. Retain the live owner handle through process-group + cleanup; persisted PIDs never authorize restart-time signals or replay. +- MLX-LM's final in-loop validation precedes its last update. Verify the actual + optimizer update count, saved adapter tensor inventory, finite parameter delta + and a separate evaluation after reloading the saved final adapter. Never + report upstream's last in-loop loss as final-adapter validation. +- Native public composition uses `native_mlx_planning.py` and + `native_mlx_family.py` through the existing `ProviderFamilyV1` and reference + host. Keep its independently authenticated hardware qualification gate closed + until real evidence exists; CPU facade tests substitute the ML child. + Host grant binding retains exact signed native command material using the + existing record port. An uncertain consumed process claim never authorizes + replay. Public logs/cancel/reconcile remain disabled. Preserve bounded + synchronous submission unless a separately reviewed owned asynchronous + lifecycle also preserves thread-affine host stores. +- Isolated native children use `-I -S -B`, explicit captured import roots and + source-byte verification before ML imports. Recompile every public SFT control; + reject unsupported controls rather than silently dropping them. Native model + roles contain deterministic MLX adapter archives, never mislabeled PEFT. + Verify tokenizer bytes, retained finite checkpoints, input binding and saved + final-adapter evidence before signing public artifact verification. ## HF Jobs diff --git a/Trainers/mlx_sft_mac/runtime-candidate/sft-qualification-proposal.json b/Trainers/mlx_sft_mac/runtime-candidate/sft-qualification-proposal.json new file mode 100644 index 00000000..0955eb02 --- /dev/null +++ b/Trainers/mlx_sft_mac/runtime-candidate/sft-qualification-proposal.json @@ -0,0 +1,77 @@ +{ + "schema": "synaptic-native-qualification-proposal/v1", + "status": "ROOT_REVIEW_REQUIRED_NOT_EXECUTION_AUTHORITY", + "profile_ref": "native-mlx-qwen3-06b-4bit-m4-candidate", + "method": "sft", + "source_requirement": "Exact independently reviewed pushed feature/integration commit; root issues the source and attempt binding after review.", + "runtime": { + "python_version": "3.11.14", + "mlx_version": "0.32.2", + "mlx_lm_version": "0.32.2", + "dependency_lock_sha256": "f9a6298260cd17f2c3d567ae3674108a71efa68e99369906487a4d8c0feb7117", + "hardware": "Apple M4", + "physical_memory_bytes": 25769803776, + "capture_requirement": "Exact source, interpreter/framework, stdlib, 50-package candidate inventory, static native dependency graph, startup files, OS build and selected-wheel provenance; no self-issued promotion." + }, + "model": { + "repository": "mlx-community/Qwen3-0.6B-4bit", + "revision": "73e3e38d981303bc594367cd910ea6eb48349da8", + "tokenizer_repository": "mlx-community/Qwen3-0.6B-4bit", + "tokenizer_revision": "73e3e38d981303bc594367cd910ea6eb48349da8", + "download_budget_bytes": 536870912, + "trust_remote_code": false, + "snapshot_requirement": "Fresh private snapshot with reviewed complete file hashes, retained root descriptors, no model conversion or shared-cache writes." + }, + "training": { + "batch_size": 1, + "max_seq_length": 128, + "max_batch_tokens": 128, + "iterations": 3, + "seed": 17, + "steps_per_report": 1, + "steps_per_eval": 3, + "steps_per_save": 3 + }, + "lora": { + "layers": 2, + "rank": 4, + "scale": 8, + "dropout": 0, + "targets": ["self_attn.q_proj", "self_attn.v_proj"] + }, + "optimizer": { + "learning_rate": 0.0001, + "beta1": 0.9, + "beta2": 0.999, + "epsilon": 0.00000001, + "weight_decay": 0, + "bias_correction": true, + "schedule": "constant" + }, + "bounds": { + "maximum_training_attempts": 2, + "maximum_total_optimizer_updates": 6, + "timeout_seconds_per_attempt": 300, + "termination_grace_seconds": 5, + "maximum_output_bytes_per_stream": 1048576, + "maximum_artifact_file_bytes": 67108864, + "maximum_artifact_set_bytes": 134217728 + }, + "input_review": { + "source": "Existing prepared-publication CLI/service, synthetic material only", + "rows_per_attempt": {"train": 3, "validation": 2}, + "attempt_1": "Raw text, full_sequence, full_conversation, no template arguments", + "attempt_2": "Prepared prompt/completion, assistant_only, prompt_completion, enable_thinking=false", + "checks": ["Exact tokenizer tokens and splits", "Causal loss against independent reference", "Noncontiguous ignored-label fixture", "No truncation or row dropping"] + }, + "acceptance": [ + "One fresh host-owned attempt per run; preserve every consumed claim", + "Actual optimizer step count equals three", + "Finite nonzero saved-adapter parameter delta", + "Held-out evaluation after reloading the saved final adapter", + "Independent native safetensors inventory and authenticated public artifact readback", + "Real Darwin deadline, stop and owner-loss/restart evidence", + "Syntunia uses TrainingAPI/RunsAPI and host-owned existing ports", + "No native GRPO or hardware capability promotion from CPU contracts" + ] +} diff --git a/docs/plans/native-desktop-training-api-v1.md b/docs/plans/native-desktop-training-api-v1.md index 0a212aeb..79a4ac37 100644 --- a/docs/plans/native-desktop-training-api-v1.md +++ b/docs/plans/native-desktop-training-api-v1.md @@ -197,3 +197,134 @@ The checked-in requirements lock and its pins are unchanged. CPU contract tests prove projection/control behavior and scalar loss algebra, not MLX graph compilation, Metal memory capacity, model compatibility, native artifact validity, process lifecycle, or successful training. + +## Execution-boundary continuation — 2026-10-20 + +The next reviewable source increment adds: + +- `native_files.py`, `native_macho.py`, `native_mlx_material.py` and the checked-in + capture CLI: descriptor-held native file inventory, bounded arm64 Mach-O static + dependency parsing, explicit Homebrew alias/OS-shared-cache treatment and + comparison against a host-reviewed capture digest. This is observed installed + material, not independent wheel-origin or dynamic-loader attestation. +- `native_mlx_input.py`: retained prepared-lease hash/size/identity verification, + shared tokenization over the exact declared rows, and private local tokenizer + snapshot binding. Semantic publication provenance remains host-owned. +- `native_mlx_composition.py` and `native_mlx_artifacts.py`: explicit LoRA and + constant-AdamW settings, realized target checks, actual optimizer count, + independent native safetensors/delta verification, and saved-final-adapter + reload/evaluation. The upstream final in-loop validation is not reused. +- `native_process.py` and `native_process_guardian.py`: existing host-store + claim/spawn/outcome retention, a fresh owned worker group, bounded time/output/ + file sizes and owner-pipe loss/cancellation. Numeric PIDs remain non-authorizing. + Mac environment inspection confirmed that this CPython lacks `os.waitid` but + exposes kqueue; the Darwin branch uses process-exit notifications without + reaping before group cleanup. See Apple's + [kqueue process-filter contract](https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/kqueue.2.html). + +### Required continuation after independent review + +1. Review the new runtime/filesystem/process boundaries and exact native profile + before any ML import. Retain the original candidate environment. Reconcile + selected wheel archives to the installation evidence; a newly captured file + digest alone must not become provenance approval. Exercise the capture CLI + on the Mac against an isolated reviewed source checkout. +2. Independently review the now implemented isolated child/bootstrap and public + input recompilation. Review actual imported module roots on Darwin, including + the Python framework and static native dependency closure. A caller-provided + workload digest or a direct internal call remains non-authorizing. +3. Review the `native_environment` representation added to existing `RuntimeSpec` + and the native `ProviderFamilyV1` described below. The Modal lock maintainer + refreshed only the existing `contracts.py` member hash and dependent skill + lock digest; inventory/non-hash pins remain unchanged. +4. Independently review the five native artifact role projections and exact + known-run reader. Worker result JSON and process exit alone are not public + completion evidence. Public cancellation remains disabled; qualify it + separately from the guardian's internal stop/owner-loss behavior. +5. Root reviews the concrete + [bounded qualification proposal](../../Trainers/mlx_sft_mac/runtime-candidate/sft-qualification-proposal.json). + The proposed model/tokenizer pin is the repository's exact + [73e3e38 revision](https://huggingface.co/mlx-community/Qwen3-0.6B-4bit/tree/73e3e38d981303bc594367cd910ea6eb48349da8); + it is a candidate, not qualified MLX-LM 0.32.2 compatibility. Two fresh + synthetic attempts allow at most six total optimizer updates and ten minutes + of worker time. Root owns model preparation, ML imports and live execution. +6. Only independent real Mac execution, final adapter/artifact verification and + Syntunia public-API readback can open native SFT capability. Native GRPO + remains a separate unavailable method. + +No native runtime/provider has been globally registered, no Mac training has +run in this implementation task, and no capability has been promoted. + +## Public family and isolated child increment — 2026-10-20 + +The implementation now contains a candidate path through the existing public +facades, exercised with a substituted ML child in CPU tests: + +1. `native_mlx_compilation.compile_native_sft` projects public `TrainingInputV1`, + prepared identity and a declarative profile into `CompiledWorkload`. + `parse_native_workload` re-derives every projected field. +2. `native_mlx_child` runs only under `-I -S -B`, restricts import roots to the + captured inventory, verifies loaded origins and recompiles pinned Python + source. It joins the exact staged source/snapshots/capture/lock to tokenization + and model training. It writes the actual input-binding document and workload. +3. `native_mlx_planning.NativePlanningV1` provides immutable coordinator context, + preflight, preparation and payloads for one host-resolved request/run. + `RuntimeSpec` represents native bytes without a container placeholder. +4. `native_mlx_family.native_mlx_family` builds the existing `ProviderFamilyV1`. + It verifies host-authenticated command material, uses `NativeOwnedProcess` + for bounded synchronous submission, reconstructs exact retained Foundation + evidence on reads, and supplies native observation/verification/streaming. + It does not create an engine database or another public training API. +5. `NativeCommandCatalogV1` retains signed exact command material through the + host record port. `NativeMaterialAuthority` and `NativeEvidenceAuthority` + reuse the host HMAC implementation with separate purposes. The consumer + resolves keys and grants; none enters the child. +6. `native_mlx_publication` verifies exact output inventory, bound source/ + tokenizer/workload/runtime, actual update count and numeric saved-adapter + delta, finite retained checkpoints and saved-final validation identity. + The five public roles contain deterministic MLX adapter/tokenizer archives + and exact workload/lineage/metrics. Archives are never labeled PEFT. + +Submission is deliberately synchronous within the declared wall-clock bound. +This preserves caller-thread ownership of host SQLite/record stores; it is not +an asynchronous desktop job service. An interrupted or unacknowledged submit +remains consumed and requires diagnosis. Public logs/cancel/reconcile are +unavailable; no restart-time signal authority comes from a numeric PID. + +### Host assembly and remaining acceptance + +The family is a library adapter, not an operational Mac consumer or an automatic +qualification issuer. Root/consumer work must still: + +- Bind the actual host-resolved source evidence and prepared publication to + the public resolved request, compiled native profile and source capture. + Use the existing `ReferenceRequestPortsV1` loader/resolver/run identity. + The engine must not fabricate a source attestation or accept a test fixture. +- Prepare the original pinned candidate and fresh owner-private model/tokenizer + snapshots, request root and empty output on the execution machine. Run the + checked-in runtime capture CLI and retain independent selected-wheel/install + provenance. Add the reusable consumer preparation/qualification workflow + before effects; there is no ad hoc operational script exemption. +- In the host `GrantAuthorityPort.bind` hook, validate source/hardware/admission, + construct an exact `NativeInvocation` bound to that Foundation command, + issue `NativeCommandMaterialV1`, and publish it through + `NativeCommandCatalogV1`. A signature issued without these checks supplies no + independent source or hardware evidence. +- Compose `native_mlx_family` with that catalog, its authority, host records, + reader evidence authority, reference Foundation/assessment authenticators and + clock; pass it to existing `compose_reference_host`. Test + `training.load/resolve/plan/preflight/start`, then `runs.outcome/verify`, + authenticated artifact streaming and host reconstruction over the same stores. +- Complete the bounded root-reviewed hardware attempts and Darwin lifecycle + evidence. A `NativeQualificationV1` is a host-authenticated commitment to + profile/source/runtime/current hardware and two distinct retained attempts. + No accepting authority is shipped. Its authority must independently verify + provenance and supporting final-adapter/public-readback evidence; matching + hashes alone are insufficient. Candidate preflight and public read/stream + capabilities remain closed until that gate is satisfied. + +CPU facade coverage proves actual coordinator/Foundation/host-store joins with +the real native artifact verifier and a substituted ML child. It does not +prove native imports, Metal execution, memory capacity, Darwin cleanup, +Syntunia product integration or usable trained-model quality. The two-run +proposal remains review-required; root owns every Mac effect and live result. diff --git a/scripts/capture_native_mlx_runtime.py b/scripts/capture_native_mlx_runtime.py new file mode 100644 index 00000000..12b853e2 --- /dev/null +++ b/scripts/capture_native_mlx_runtime.py @@ -0,0 +1,78 @@ +#!/usr/bin/env python3 +"""Read-only native MLX runtime capture/check; never launches training. + +Run with the selected candidate Python and -B. Output is a new, exclusive 0600 +file. Config contains `capture` (NativeCaptureConfig) and `expectation` +(MLXRuntimeExpectation); the lock is the original reviewed requirements.lock. +""" + +from __future__ import annotations + +import argparse +import hashlib +import os +from pathlib import Path +import sys + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) + +from tuner.runtime.native_files import NativeTree +from tuner.runtime.native_mlx_material import ( + NativeCaptureConfig, capture_native_material, strict_json, verify_native_material, +) +from tuner.runtime.native_mlx_runtime import MLXRuntimeExpectation + + +def read(path, maximum): + path = Path(path) + with NativeTree(path.parent) as tree: + return tree.read(path.name, maximum=maximum) + + +def main(argv=None): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--config", required=True) + parser.add_argument("--dependency-lock", required=True) + mode = parser.add_mutually_exclusive_group(required=True) + mode.add_argument("--output") + mode.add_argument("--check") + parser.add_argument("--expected-digest") + args = parser.parse_args(argv) + try: + config = strict_json(read(args.config, 1024 * 1024)) + if type(config) is not dict or set(config) != {"capture", "expectation"}: + raise ValueError("configuration differs") + lock = read(args.dependency_lock, 1024 * 1024) + capture = NativeCaptureConfig.from_dict(config["capture"]) + expectation = MLXRuntimeExpectation.from_dict(config["expectation"]) + if args.check: + if args.expected_digest is None: + raise ValueError("check requires reviewed digest") + raw = read(args.check, 16 * 1024 * 1024) + # The supplied configuration remains binding in check mode. + actual = capture_native_material(capture, expectation, dependency_lock=lock) + if actual != raw: + raise ValueError("configuration or observed runtime differs") + verify_native_material(raw, expected_digest=args.expected_digest, dependency_lock=lock) + else: + if args.expected_digest is not None: + raise ValueError("capture cannot self-approve a digest") + raw = capture_native_material(capture, expectation, dependency_lock=lock) + destination = Path(args.output) + with NativeTree(destination.parent, private=True) as parent: + fd = os.open(destination.name, os.O_WRONLY | os.O_CREAT | os.O_EXCL + | os.O_NOFOLLOW, 0o600, dir_fd=parent.fd) + with os.fdopen(fd, "wb") as stream: + stream.write(raw) + stream.flush() + os.fsync(stream.fileno()) + parent.check() + print("OBSERVED_NOT_ADMITTED sha256=" + hashlib.sha256(raw).hexdigest()) + return 0 + except Exception: + print("NATIVE_CAPTURE_REJECTED", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/runtime/test_native_mlx_child.py b/tests/runtime/test_native_mlx_child.py new file mode 100644 index 00000000..c86db67b --- /dev/null +++ b/tests/runtime/test_native_mlx_child.py @@ -0,0 +1,173 @@ +from dataclasses import replace +import hashlib +import os +from pathlib import Path +import subprocess +import sys +import tempfile +from types import ModuleType + +import pytest + +from synaptic_tuner.api.v1.training_input import TrainingInputV1 +from synaptic_tuner.api.v1.training_sources import PreparedTrainingInputIdentity +from tuner.runtime.native_mlx_child import _PinnedFinder, run_request +from tuner.runtime.native_mlx_material import canonical +from tuner.training.native_mlx_compilation import NativeMLXProfile, compile_native_sft, parse_native_workload + + +def inputs(): + identity = PreparedTrainingInputIdentity( + "prepared://sha256/" + "a" * 64, "a" * 64, "b" * 64, 100, "syntunia-sft-row/v1", + ) + training = TrainingInputV1.from_dict({ + "schema_version": "synaptic-training-input/v1", "method": "sft", + "model": {"ref": "configured/model", "revision": "c" * 40, "tokenizer_revision": "c" * 40}, + "dataset": {"ref": identity.ref}, + "hyperparameters": { + "schema_version": "synaptic-sft-hyperparameters/v1", + "batch_size": 1, "gradient_accumulation_steps": 1, "learning_rate": .0001, + "duration": {"max_steps": 3, "num_epochs": None}, "max_seq_length": 128, "seed": 17, + "save_steps": 3, "save_total_limit": 1, "lora_rank": 4, "lora_alpha": 32, + "lora_dropout": 0, "lora_target_modules": ["self_attn.q_proj", "self_attn.v_proj"], + "use_dora": False, "use_rslora": False, "init_lora_weights": True, + "split_dataset": False, "dataset_format": "raw_text", + "completion_only_loss": False, "assistant_only_loss": False, + "use_preassigned_splits": True, "prompt_render": "full_conversation", + "packing": False, "require_memory_efficient_loss": False, + }, + "artifacts": {"required_kinds": ["final_model", "training_lineage"], "retain_checkpoints": True}, + }) + profile = NativeMLXProfile( + "candidate", "configured/model", "c" * 40, "c" * 40, "d" * 64, "e" * 64, + 2, "alpha_over_rank", .9, .999, 1e-8, 0, True, "constant", True, 1, 3, 3, 128, + ) + return training, identity, profile + + +def test_public_input_compiles_exact_native_settings_and_reparses(): + projection = compile_native_sft(*inputs()) + assert projection.lora.scale == 8 + assert projection.controls.iterations == 3 + assert projection.optimizer.learning_rate == .0001 + assert projection.tokenization.assistant_only_loss is False + assert parse_native_workload(projection.workload.canonical_bytes) == projection + mutated = projection.workload.document + mutated["controls"]["iterations"] = 4 + with pytest.raises(ValueError, match="compilation"): + parse_native_workload(canonical(mutated)) + + +@pytest.mark.parametrize("changes", [ + {"gradient_accumulation_steps": 2}, {"use_dora": True}, {"use_rslora": True}, + {"init_lora_weights": False}, {"packing": True}, {"split_dataset": True}, + {"require_memory_efficient_loss": True}, {"use_preassigned_splits": False}, + {"duration": {"max_steps": 4, "num_epochs": None}}, + {"duration": {"max_steps": None, "num_epochs": 1}}, + {"save_steps": 1}, +]) +def test_unsupported_public_controls_are_rejected_not_silently_ignored(changes): + training, identity, profile = inputs() + document = training.to_dict() + document["hyperparameters"].update(changes) + with pytest.raises(ValueError): + compile_native_sft(TrainingInputV1.from_dict(document), identity, profile) + + +def test_profile_model_and_source_identity_do_not_float(): + training, identity, profile = inputs() + with pytest.raises(ValueError, match="model profile"): + compile_native_sft(training, identity, replace(profile, model_revision="f" * 40)) + with pytest.raises(ValueError, match="publication"): + compile_native_sft(training, replace(identity, ref="prepared://sha256/" + "f" * 64, + revision="f" * 64), profile) + with pytest.raises(ValueError, match="alpha_over_rank"): + replace(profile, lora_scale_policy="guess") + + +def test_public_completion_controls_project_to_shared_mask_operation(): + training, identity, profile = inputs() + document = training.to_dict() + document["hyperparameters"].update({ + "dataset_format": "messages", "completion_only_loss": True, + "assistant_only_loss": False, "prompt_render": "prompt_completion", + "chat_template_kwargs": {"enable_thinking": False}, + }) + projection = compile_native_sft( + TrainingInputV1.from_dict(document), replace(identity, format="syntunia-sft-row/v2"), profile, + ) + assert projection.tokenization.assistant_only_loss is True + assert projection.tokenization.chat_template_json == b'{"enable_thinking":false}' + + +@pytest.fixture +def private_root(): + mask = os.umask(0o077) + try: + with tempfile.TemporaryDirectory(prefix="native-child-test-", dir=Path.home()) as path: + yield Path(path) + finally: + os.umask(mask) + + +def test_import_finder_rejects_ambient_sibling_and_compiles_only_pinned_source(private_root): + source = private_root / "owned_fixture.py" + source.write_text("result = 17\n") + finder = _PinnedFinder({str(source): hashlib.sha256(source.read_bytes()).hexdigest()}) + spec = finder.find_spec("owned_fixture", [str(private_root)]) + module = ModuleType("owned_fixture") + spec.loader.exec_module(module) + assert module.result == 17 + (private_root / "ambient_fixture.py").write_text("result = 99\n") + with pytest.raises(ImportError, match="outside capture"): + finder.find_spec("ambient_fixture", [str(private_root)]) + source.write_text("result = 23\n") + with pytest.raises(ImportError, match="source bytes"): + spec.loader.exec_module(module) + + +def test_worker_recompiles_and_joins_workload_before_invoking_runtime(private_root, monkeypatch): + from tuner.runtime import native_mlx_input, native_mlx_composition + projection = compile_native_sft(*inputs()) + output = private_root / "output" + output.mkdir(mode=0o700) + for name, content in (("capture.json", b"capture"), ("lock", b"lock"), ("dataset.jsonl", b"dataset")): + (private_root / name).write_bytes(content) + snapshot = { + "repository": "configured/model", "revision": "c" * 40, + "root": str(private_root / "model"), + "files": [{"path": "config.json", "size": 2, "sha256": "f" * 64}], + } + request = { + "schema": "synaptic-native-child-request/v1", "workload": projection.workload.document, + "runtime_capture": str(private_root / "capture.json"), + "dependency_lock": str(private_root / "lock"), "dataset": str(private_root / "dataset.jsonl"), + "model_snapshot": snapshot, "tokenizer_snapshot": snapshot, "output": str(output), + } + observed = {} + + def prepare(**kwargs): + observed.update(kwargs) + return "prepared-fixture" + + def execute(**kwargs): + assert kwargs["bound"] == "prepared-fixture" + assert kwargs["lora"] == projection.lora and kwargs["optimizer"] == projection.optimizer + return b"verified-fixture-result" + + monkeypatch.setattr(native_mlx_input, "prepare_native_input", prepare) + monkeypatch.setattr(native_mlx_composition, "execute_native_sft", execute) + assert run_request(request, request_root=private_root) == b"verified-fixture-result" + assert observed["workload_digest"] == projection.workload.fingerprint + assert observed["runtime_digest"] == projection.profile.runtime_digest + assert (output / "workload.json").read_bytes() == projection.workload.canonical_bytes + + +def test_child_requires_isolation_and_has_closed_diagnostics(private_root): + from tuner.runtime import native_mlx_child + path = Path(native_mlx_child.__file__) + result = subprocess.run( + [sys.executable, str(path), "--request", str(private_root / "missing")], + capture_output=True, timeout=5, + ) + assert result.returncode == 120 and result.stdout == result.stderr == b"" diff --git a/tests/runtime/test_native_mlx_execution.py b/tests/runtime/test_native_mlx_execution.py new file mode 100644 index 00000000..14cdd860 --- /dev/null +++ b/tests/runtime/test_native_mlx_execution.py @@ -0,0 +1,304 @@ +from dataclasses import replace +import hashlib +import io +import json +from pathlib import Path +import struct +import subprocess +import sys +import tempfile +import os +from types import SimpleNamespace, ModuleType + +import pytest + +from synaptic_tuner.api.v1.training_sources import PreparedTrainingInputIdentity +from tuner.runtime.native_files import NativeFile +from tuner.runtime.native_mlx_artifacts import adapter_tensors, verify_adapter_delta +from tuner.runtime.native_mlx_composition import NativeAdamW, NativeLoRA, attach_native_lora +from tuner.runtime.native_mlx_input import ( + NativeModelSnapshot, NativeTokenization, read_retained_native_source, tokenize_native_source, +) +from tuner.runtime.native_mlx_sft import MLXSFTConfig +from tuner.training.contracts import RetainedTrainingInputStreamLease + + +class Source: + def __init__(self, raw, *, lease_identity=None): + self.raw = raw + digest = hashlib.sha256(raw).hexdigest() + self.identity = PreparedTrainingInputIdentity( + "prepared://sha256/" + "a" * 64, "a" * 64, digest, len(raw), "syntunia-sft-row/v1", + ) + self.lease_identity = lease_identity + self.last_stream = None + + def open_lease(self): + self.last_stream = io.BytesIO(self.raw) + return RetainedTrainingInputStreamLease(self.lease_identity or self.identity, self.last_stream) + + +def source(): + return Source(b"".join(json.dumps({ + "schema_version": "syntunia-sft-row/v1", "format": "raw_text", + "row_id": row_id, "text": text, "split": split, + }).encode() + b"\n" for row_id, text, split in [ + ("train-1", "ab", "train"), ("val-1", "cd", "validation"), + ("train-2", "ef", "train"), ("val-2", "g", "validation"), + ])) + + +class Tokenizer: + eos_token_id = 0 + + def encode(self, text, add_special_tokens=False): + assert add_special_tokens is False + return [ord(c) for c in text] + + +def tokenize(prepared_source): + return tokenize_native_source( + source=prepared_source, identity=prepared_source.identity, tokenizer=Tokenizer(), + tokenizer_snapshot=NativeModelSnapshot( + "configured/model", "c" * 40, "/private/configured-model", + (NativeFile("tokenizer.json", 3, "d" * 64),), + ), + tokenization=NativeTokenization(False, "full_conversation", None, True), + config=MLXSFTConfig(2, 16, 32, 3, 1, 1, 1, 3), + workload_digest="e" * 64, runtime_digest="f" * 64, + ) + + +def test_retained_bytes_are_shared_tokenized_without_resplit_or_second_eos(): + retained = source() + bound = tokenize(retained) + assert tuple(row.row_id for row in bound.prepared.train) == ("train-1", "train-2") + assert tuple(row.row_id for row in bound.prepared.validation) == ("val-1", "val-2") + assert bound.prepared.train[0].input_ids == (97, 98, 0) + assert bound.prepared.train[0].labels == (97, 98, 0) + assert retained.last_stream.closed + assert len(bound.digest) == 64 + assert replace(bound, workload_digest="0" * 64).digest != bound.digest + + +def test_lease_identity_and_source_bytes_reject_before_tokenization(): + retained = source() + retained.raw += b" " + with pytest.raises(ValueError, match="bytes differ"): + tokenize(retained) + assert retained.last_stream.closed + retained = source() + retained.lease_identity = replace(retained.identity, content_digest="b" * 64) + with pytest.raises(ValueError, match="lease identity"): + read_retained_native_source(retained, retained.identity) + assert retained.last_stream.closed + + +def test_unknown_split_and_duplicate_row_are_not_repaired(): + retained = source() + with pytest.raises(ValueError, match="split"): + tokenize(Source(retained.raw.replace(b'"validation"', b'"test"'))) + with pytest.raises(ValueError, match="more than once"): + tokenize(Source(retained.raw.replace(b'"val-1"', b'"train-1"'))) + + +def test_native_modules_keep_sdk_imports_lazy(): + result = subprocess.run( + [sys.executable, "-c", + "import sys; import tuner.runtime.native_mlx_composition; " + "import tuner.training.native_mlx_family; import tuner.training.native_mlx_planning; " + "assert not any(n.split('.')[0] in " + "{'mlx','mlx_lm','transformers','torch','numpy','modal'} for n in sys.modules)"], + cwd=Path(__file__).resolve().parents[2], capture_output=True, timeout=15, + ) + assert result.returncode == 0, result.stderr + + +def tensor_bytes(values, *, dtype="F32", name="layers.0.q_proj.lora_a", shape=None): + shape = shape or [1, len(values)] + if dtype == "BF16": + body = b"".join(struct.pack("> 16) + for v in values) + else: + body = struct.pack("<" + ("f" if dtype == "F32" else "e") * len(values), *values) + header = json.dumps({name: { + "dtype": dtype, "shape": shape, "data_offsets": [0, len(body)], + }}, separators=(",", ":")).encode() + return struct.pack(" 0 + assert recompose().api().runs.reverify(started.run).verified + assert len(calls) == 1 + with pytest.raises(RunOperationError): + api.runs.logs(RunLogsRequest(started.run, limit=1, maximum_bytes=4096)) + + +def test_consumed_uncertain_native_submission_is_not_replayed(publication, monkeypatch): + state = prepare(publication) + composition, recompose, planning, calls, catalog = compose(state, publication, monkeypatch, fail=True) + api = composition.api() + plan = api.training.plan(api.training.resolve(api.training.load(state.training.canonical_json())), + planning.provider) + ready = api.training.preflight(plan) + start = api.training.start(plan, ready) + assert api.runs.show(start.run).state.value == "reconcile_required" + recompose().api().training.start(plan, ready) + assert len(calls) == 1 + + +def test_native_catalog_rejects_forged_envelope_and_authenticated_collision(publication, monkeypatch): + state = prepare(publication) + composition, _, planning, _, catalog = compose(state, publication, monkeypatch) + api = composition.api() + plan = api.training.plan(api.training.resolve(api.training.load(state.training.canonical_json())), + planning.provider) + started = api.training.start(plan, api.training.preflight(plan)) + workflow = composition.stores.workflow_store.get(started.run) + material = catalog.resolve(workflow.submit.command_digest) + assert NativeCommandMaterialV1.parse(material.canonical_bytes) == material + with pytest.raises(ValueError, match="unauthenticated"): + catalog.publish_once(replace(material, tag="0" * 64)) + changed = catalog.authority.issue(replace(material, tokenizer_members=("different.json",))) + with pytest.raises(ValueError, match="collision"): + catalog.publish_once(changed) + + +def test_public_native_reverification_detects_changed_saved_adapter(publication, monkeypatch): + state = prepare(publication) + composition, _, planning, calls, _ = compose(state, publication, monkeypatch) + api = composition.api() + plan = api.training.plan(api.training.resolve(api.training.load(state.training.canonical_json())), + planning.provider) + started = api.training.start(plan, api.training.preflight(plan)) + api.runs.outcome(started.run) + assert api.runs.verify(started.run).verified + (publication["output"] / "adapters.safetensors").write_bytes(b"corrupt") + with pytest.raises(RunOperationError): + api.runs.reverify(started.run) + assert len(calls) == 1 + + +def test_native_qualification_is_bound_and_authority_failure_closes_preflight(publication): + state = prepare(publication) + qualified = qualified_planning(state) + proof = replace(qualified._qualification, runtime_digest="9" * 64) + rejected = NativePlanningV1( + **state.kwargs, qualification=proof, qualification_authority=TestQualificationAuthority(), + ) + assert not rejected.preflight(rejected.plan).ready + assert not rejected.descriptor.capabilities.artifact_streaming + + +def test_unsigned_process_record_tampering_cannot_change_authenticated_terminal_outcome(publication, monkeypatch): + state = prepare(publication) + composition, _, planning, calls, catalog = compose(state, publication, monkeypatch) + api = composition.api() + plan = api.training.plan(api.training.resolve(api.training.load(state.training.canonical_json())), + planning.provider) + started = api.training.start(plan, api.training.preflight(plan)) + workflow = composition.stores.workflow_store.get(started.run) + material = catalog.resolve(workflow.submit.command_digest) + key = f"native-mlx/{material.invocation.attempt}/outcome" + retained = catalog.records.read(partition="effects", key=key) + changed = strict_json(retained.canonical) + changed["value"]["exit_code"] = 9 + assert catalog.records.compare_and_swap( + partition="effects", key=key, expected_revision=retained.revision, canonical=canonical(changed), + ) + with pytest.raises(RunOperationError): + api.runs.outcome(started.run) + assert len(calls) == 1 diff --git a/tests/runtime/test_native_mlx_material.py b/tests/runtime/test_native_mlx_material.py new file mode 100644 index 00000000..3c2d9e54 --- /dev/null +++ b/tests/runtime/test_native_mlx_material.py @@ -0,0 +1,243 @@ +from dataclasses import asdict, replace +import hashlib +import json +import os +from pathlib import Path +import struct +import tempfile + +import pytest + +from tuner.runtime.native_files import NativeTree +from tuner.runtime.native_macho import macho_links, resolve_links +from tuner.runtime import native_mlx_material as material +from tuner.runtime.native_mlx_runtime import MLXRuntimeExpectation + + +@pytest.fixture +def private_tree(): + # Real runtime roots intentionally reject /tmp's public writable ancestor. + previous = os.umask(0o077) + try: + with tempfile.TemporaryDirectory(prefix="native-contract-", dir=Path.home()) as value: + yield Path(value) + finally: + os.umask(previous) + + +def macho(*dependencies, rpaths=()): + commands = [] + for command, name, minimum in [ + *((0xC, name, 24) for name in dependencies), + *((0x8000001C, name, 12) for name in rpaths), + ]: + value = name.encode() + b"\0" + size = (minimum + len(value) + 7) // 8 * 8 + commands.append(struct.pack("I", 1) + raw += struct.pack(">IIIII", 0x0100000C, 0, 28, len(arm), 0) + arm + assert macho_links(raw).dependencies == ("/usr/lib/libSystem.B.dylib",) + with pytest.raises(ValueError): + macho_links(raw[:-1]) + + +def test_native_dependency_resolution_requires_captured_target(private_tree): + executable = private_tree / "python" + library = private_tree / "Python" + executable.write_bytes(macho("@loader_path/Python")) + library.write_bytes(macho("/usr/lib/libSystem.B.dylib")) + links = {str(p): macho_links(p.read_bytes()) for p in (executable, library)} + args = dict(executable=str(executable), system_libraries=("/usr/lib/libSystem.B.dylib",)) + with pytest.raises(ValueError, match="outside captured"): + resolve_links(links, admitted_files={str(executable)}, **args) + edges = resolve_links(links, admitted_files=set(links), **args) + assert (str(executable), "@loader_path/Python", str(library)) in edges + + +@pytest.fixture +def capture(private_tree, monkeypatch): + trees = [private_tree / name for name in ("stdlib", "packages", "source")] + for path in trees: + path.mkdir() + (path / "module.py").write_text("# pinned\n") + executable = private_tree / "python" + library = private_tree / "Python" + executable.write_bytes(macho("@loader_path/Python")) + library.write_bytes(macho("/usr/lib/libSystem.B.dylib")) + lock = b"reviewed-original-hash-lock" + expected = MLXRuntimeExpectation( + "3.11.14", hashlib.sha256(executable.read_bytes()).hexdigest(), + hashlib.sha256(lock).hexdigest(), (("mlx", "0.32.2"), ("mlx-lm", "0.32.2")), + ) + config = material.NativeCaptureConfig( + str(trees[0]), str(trees[1]), (str(trees[2]),), + tuple(sorted((str(executable), str(library)))), ("/usr/lib/libSystem.B.dylib",), + ) + runtime = asdict(expected) + runtime["distributions"] = dict(expected.distributions) + runtime["status"] = "CANDIDATE_MATCH_ONLY" + monkeypatch.setattr(material.sys, "executable", str(executable)) + monkeypatch.setattr(material.sysconfig, "get_path", lambda key: str(trees[0])) + monkeypatch.setattr(material, "inspect_mlx_runtime_candidate", lambda *a, **kw: runtime) + from importlib import metadata + + class Distribution: + def locate_file(self, _): + return trees[1] + monkeypatch.setattr(metadata, "distributions", lambda: [Distribution()]) + return config, expected, lock, library + + +def test_material_rechecks_framework_not_only_python_stub(capture): + config, expected, lock, framework = capture + raw = material.capture_native_material(config, expected, dependency_lock=lock) + digest = hashlib.sha256(raw).hexdigest() + material.verify_native_material(raw, expected_digest=digest, dependency_lock=lock) + framework.write_bytes(framework.read_bytes() + b"tampered native bytes") + with pytest.raises(ValueError, match="runtime bytes"): + material.verify_native_material(raw, expected_digest=digest, dependency_lock=lock) + + +def test_material_rechecks_added_importable_file_and_startup(capture): + config, expected, lock, _ = capture + raw = material.capture_native_material(config, expected, dependency_lock=lock) + (Path(config.packages) / "unreviewed.pth").write_text("import unexpected") + with pytest.raises(ValueError): + material.verify_native_material(raw, expected_digest=hashlib.sha256(raw).hexdigest(), + dependency_lock=lock) + + +def test_canonical_capture_and_host_digest_are_required(capture): + config, expected, lock, _ = capture + raw = material.capture_native_material(config, expected, dependency_lock=lock) + with pytest.raises(ValueError, match="digest"): + material.verify_native_material(raw, expected_digest="0" * 64, dependency_lock=lock) + pretty = json.dumps(json.loads(raw), indent=2).encode() + with pytest.raises(ValueError, match="canonical"): + material.verify_native_material(pretty, expected_digest=hashlib.sha256(pretty).hexdigest(), + dependency_lock=lock) + + +def test_homebrew_aliases_bind_spelling_and_captured_target(capture): + config, expected, lock, framework = capture + alias = Path(config.stdlib) / "framework.dylib" + alias.symlink_to(framework) + config = replace(config, aliases=((str(alias), str(framework)),)) + raw = material.capture_native_material(config, expected, dependency_lock=lock) + assert json.loads(raw)["aliases"][0]["disposition"] == "captured-file-alias" + alias.unlink() + alias.symlink_to(Path(config.packages) / "module.py") + with pytest.raises(ValueError, match="captured file"): + material.verify_native_material(raw, expected_digest=hashlib.sha256(raw).hexdigest(), + dependency_lock=lock) + + +def test_global_site_packages_alias_is_pinned_but_not_followed(capture): + config, expected, lock, framework = capture + global_packages = framework.parent / "global-packages" + global_packages.mkdir(mode=0o700) + (global_packages / "must_not_capture.py").write_text("not on isolated child path") + alias = Path(config.stdlib) / "site-packages" + alias.symlink_to(global_packages) + config = replace(config, aliases=((str(alias), str(global_packages)),)) + raw = material.capture_native_material(config, expected, dependency_lock=lock) + parsed = json.loads(raw) + assert parsed["aliases"][0]["disposition"] == "excluded-global-site-packages" + assert "must_not_capture.py" not in raw.decode() + + +def test_existing_runtime_spec_represents_native_without_invented_oci(capture): + from tuner.training.contracts import RuntimeSpec + config, expected, lock, _ = capture + captured = json.loads(material.capture_native_material(config, expected, dependency_lock=lock)) + captured["platform"] = { + "system": "Darwin", "machine": "arm64", "macos": "15.0", + "kernel_build": "reviewed-kernel-build", + } + raw = material.canonical(captured) + spec = material.native_runtime_spec(raw) + assert spec.image is None and spec.material_kind == "native_environment" + assert spec.material_digest == hashlib.sha256(raw).hexdigest() + assert spec.dependency_lock_digest == expected.dependency_lock_sha256 + with pytest.raises(ValueError): + RuntimeSpec("repo/image@sha256:" + "a" * 64, "b" * 64, "3.11.14", + "native_environment", "c" * 64) + captured["platform"]["system"] = "Linux" + with pytest.raises(ValueError, match="platform"): + material.native_runtime_spec(material.canonical(captured)) + + +@pytest.mark.parametrize("raw", [b'{"a":1,"a":2}', b'{"a":NaN}', b'{"a":Infinity}']) +def test_strict_capture_parser(raw): + with pytest.raises(ValueError): + material.strict_json(raw) diff --git a/tests/runtime/test_native_mlx_publication.py b/tests/runtime/test_native_mlx_publication.py new file mode 100644 index 00000000..7c47dbaa --- /dev/null +++ b/tests/runtime/test_native_mlx_publication.py @@ -0,0 +1,167 @@ +from dataclasses import asdict +import hashlib +import io +import os +from pathlib import Path +import struct +import tarfile +import tempfile + +import pytest + +from tests.runtime.test_native_mlx_child import inputs +from tuner.runtime.native_files import NativeTree +from tuner.runtime.native_mlx_artifacts import adapter_tensors, verify_adapter_delta +from tuner.runtime.native_mlx_input import NativeModelSnapshot +from tuner.runtime.native_mlx_material import canonical, strict_json +from tuner.runtime.native_mlx_publication import NativeArtifactBudget, verify_native_publication +from tuner.training.native_mlx_compilation import compile_native_sft + + +def tensor_archive(value): + headers, payload = {}, bytearray() + for layer in range(2): + for target in ("q_proj", "v_proj"): + for suffix in ("lora_a", "lora_b"): + name = f"model.layers.{layer}.self_attn.{target}.{suffix}" + start = len(payload) + payload.extend(struct.pack("<4f", value, 0, 0, 0)) + headers[name] = {"dtype": "F32", "shape": [1, 4], + "data_offsets": [start, len(payload)]} + header = canonical(headers) + return struct.pack(" 0 + assert process.observe(invocation)["state"] == "terminated" + assert NativeOwnedProcess(records).cancel(invocation) is False + with pytest.raises(ValueError, match="already exists"): + process.run_once(invocation) + + +@pytest.mark.parametrize("script,reason", [ + ("import time; time.sleep(60)", "deadline"), + ("import os\nwhile True: os.write(1, b'x' * 65536)", "output_limit"), + ("import os\nwhile True: os.write(2, b'x' * 65536)", "output_limit"), +]) +def test_deadline_and_output_are_actual_process_bounds(invocation, script, reason): + invocation = worker(invocation, script) + outcome = NativeOwnedProcess(InMemoryDurableRecordStoreV1()).run_once(invocation) + assert outcome["reason"] == reason + assert outcome["termination_confirmed"] is True and outcome["exit_code"] != 0 + + +def test_current_owner_can_cancel_without_replaying_numeric_pid(invocation): + invocation = worker(invocation, "import time; time.sleep(60)") + process = NativeOwnedProcess(InMemoryDurableRecordStoreV1()) + outputs = [] + thread = threading.Thread(target=lambda: outputs.append(process.run_once(invocation))) + thread.start() + deadline = time.monotonic() + 2 + accepted = False + while time.monotonic() < deadline: + if process.cancel(invocation): + accepted = True + break + time.sleep(.01) + thread.join(timeout=6) + assert accepted and not thread.is_alive() + assert outputs[0]["reason"] == "cancel_requested" + assert process.cancel(invocation) is False + + +def test_spawn_retention_failure_preserves_uncertain_claim(invocation): + class FailingStore(InMemoryDurableRecordStoreV1): + def create(self, *, partition, key, canonical): + if key.endswith("/spawn"): + raise RuntimeError("simulated interrupted storage") + return super().create(partition=partition, key=key, canonical=canonical) + records = FailingStore() + process = NativeOwnedProcess(records) + with pytest.raises(RuntimeError): + process.run_once(invocation) + assert process.observe(invocation) == {"state": "uncertain", "automatic_replay": False} + with pytest.raises(ValueError, match="already exists"): + NativeOwnedProcess(records).run_once(invocation) + + +def test_byte_drift_rejects_before_claim(invocation): + Path(invocation.worker).write_text("changed") + process = NativeOwnedProcess(InMemoryDurableRecordStoreV1()) + with pytest.raises(ValueError, match="bytes differ"): + process.run_once(invocation) + assert process.observe(invocation) == {"state": "unclaimed"} + + +def test_guardian_stops_worker_after_owner_pipe_eof(invocation): + invocation = worker(invocation, "import time; time.sleep(60)") + read_fd, write_fd = os.pipe() + proc = subprocess.Popen( + [sys.executable, "-I", "-S", "-B", invocation.guardian, str(read_fd)], + stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, + pass_fds=(read_fd,), + ) + os.close(read_fd) + spec = { + "argv": [sys.executable, "-I", "-S", "-B", invocation.worker], + "cwd": invocation.private_root, "environment": {}, + "timeout_seconds": 10, "termination_grace_seconds": 1, + "maximum_output_bytes": 1024, "maximum_file_bytes": 1024**2, + } + os.close(write_fd) + out, err = proc.communicate(json.dumps(spec).encode() + b"\n", timeout=5) + assert proc.returncode == 0, err + outcome = json.loads(out) + assert outcome["reason"] == "owner_lost" and outcome["termination_confirmed"] is True + + +def test_darwin_exit_observer_uses_kqueue_without_reaping(monkeypatch): + calls = [] + + class Queue: + def control(self, changes, maximum, timeout): + calls.append(changes) + if len(calls) == 1: + return [] + return [type("Event", (), {"flags": 0, "ident": 42, "fflags": 4})()] + + def close(self): + calls.append("closed") + monkeypatch.setattr(guardian.sys, "platform", "darwin") + monkeypatch.setattr(guardian.select, "kqueue", Queue, raising=False) + monkeypatch.setattr(guardian.select, "kevent", lambda *a, **k: (a, k), raising=False) + for name, value in (("KQ_FILTER_PROC", 1), ("KQ_EV_ADD", 1), ("KQ_EV_ONESHOT", 2), + ("KQ_NOTE_EXIT", 4), ("KQ_EV_ERROR", 8)): + monkeypatch.setattr(guardian.select, name, value, raising=False) + monkeypatch.setattr(guardian.os, "waitid", lambda *a: pytest.fail("Darwin must not call waitid")) + observer = guardian._ExitObserver(type("Child", (), {"pid": 42})()) + assert observer.exited() is True + observer.close() + assert calls[-1] == "closed" diff --git a/tuner/execution/providers/modal/inference-runtime.lock.json b/tuner/execution/providers/modal/inference-runtime.lock.json index d7d654c5..da653775 100644 --- a/tuner/execution/providers/modal/inference-runtime.lock.json +++ b/tuner/execution/providers/modal/inference-runtime.lock.json @@ -1 +1 @@ -{"base_registry_reference":"docker.io/vllm/vllm-openai@sha256:116aa00ee0b68855616a56e1d7e1ae937e591a8bd6969ee45cbcedb246ddf355","dependency_lock_path":"tuner/execution/providers/modal/inference-dependencies.lock","distributions":{"accelerate":"1.13.0","aiofile":"3.9.0","aiofiles":"25.1.0","aiohappyeyeballs":"2.6.1","aiohttp":"3.13.3","aiosignal":"1.4.0","annotated-doc":"0.0.4","annotated-types":"0.7.0","anthropic":"0.84.0","anyio":"4.12.1","apache-tvm-ffi":"0.1.9","astor":"0.8.1","attrs":"25.4.0","awscrt":"0.31.3","bitsandbytes":"0.49.2","blake3":"1.0.8","boto3":"1.42.65","botocore":"1.42.65","cachetools":"7.0.5","caio":"0.9.25","cbor2":"5.8.0","certifi":"2026.2.25","cffi":"2.0.0","charset-normalizer":"3.4.5","click":"8.3.1","cloudpickle":"3.1.2","compressed-tensors":"0.13.0","cryptography":"46.0.5","cuda-bindings":"12.9.4","cuda-pathfinder":"1.4.2","cuda-python":"12.9.4","cufile-python":"0.2.0","cupy-cuda12x":"14.0.1","deep-ep":"1.2.1+73b6ea4","deep-gemm":"2.3.0+477618c","depyf":"0.20.0","dill":"0.4.1","diskcache":"5.6.3","distro":"1.9.0","dnspython":"2.8.0","docstring-parser":"0.17.0","einops":"0.8.2","email-validator":"2.3.0","fastapi":"0.135.1","fastapi-cli":"0.0.24","fastapi-cloud-cli":"0.14.1","fastar":"0.8.0","filelock":"3.25.1","flashinfer-cubin":"0.6.4","flashinfer-jit-cache":"0.6.4+cu129","flashinfer-python":"0.6.4","frozenlist":"1.8.0","fsspec":"2026.2.0","gguf":"0.18.0","google-api-core":"2.30.0","google-auth":"2.49.0","google-cloud-core":"2.5.0","google-cloud-storage":"3.9.0","google-crc32c":"1.8.0","google-resumable-media":"2.8.0","googleapis-common-protos":"1.73.0","grpcio":"1.78.0","grpcio-reflection":"1.78.0","grpclib":"0.4.9","h11":"0.16.0","h2":"4.4.1","hf-transfer":"0.1.9","hf-xet":"1.3.2","hpack":"4.2.0","httpcore":"1.0.9","httptools":"0.7.1","httpx":"0.28.1","httpx-sse":"0.4.3","huggingface-hub":"0.36.2","humanize":"4.15.0","hyperframe":"6.1.0","idna":"3.11","ijson":"3.5.0","importlib-metadata":"8.7.1","interegular":"0.3.3","jinja2":"3.1.6","jiter":"0.13.0","jmespath":"1.1.0","jsonschema":"4.26.0","jsonschema-specifications":"2025.9.1","kaldi-native-fbank":"1.22.3","lark":"1.2.2","llguidance":"1.3.0","llvmlite":"0.44.0","lm-format-enforcer":"0.11.3","lmcache":"0.4.0","loguru":"0.7.3","markdown-it-py":"4.0.0","markupsafe":"3.0.3","mcp":"1.26.0","mdurl":"0.1.2","mistral-common":"1.9.1","modal":"1.5.4","model-hosting-container-standards":"0.1.13","modelscope":"1.34.0","mooncake-transfer-engine":"0.3.9","mpmath":"1.3.0","msgpack":"1.1.2","msgspec":"0.20.0","multidict":"6.7.1","networkx":"3.6.1","ninja":"1.13.0","nixl":"0.9.0","nixl-cu12":"0.10.1","numba":"0.61.2","numpy":"2.2.6","nvidia-cublas-cu12":"12.9.1.4","nvidia-cuda-cupti-cu12":"12.9.79","nvidia-cuda-nvrtc-cu12":"12.9.86","nvidia-cuda-runtime-cu12":"12.9.79","nvidia-cudnn-cu12":"9.10.2.21","nvidia-cudnn-frontend":"1.18.0","nvidia-cufft-cu12":"11.4.1.4","nvidia-cufile-cu12":"1.14.1.1","nvidia-curand-cu12":"10.3.10.19","nvidia-cusolver-cu12":"11.7.5.82","nvidia-cusparse-cu12":"12.5.10.65","nvidia-cusparselt-cu12":"0.7.1","nvidia-cutlass-dsl":"4.4.1","nvidia-cutlass-dsl-libs-base":"4.4.1","nvidia-ml-py":"13.590.48","nvidia-nccl-cu12":"2.27.5","nvidia-nvjitlink-cu12":"12.9.86","nvidia-nvshmem-cu12":"3.4.5","nvidia-nvtx-cu12":"12.9.79","nvtx":"0.2.14","openai":"2.24.0","openai-harmony":"0.0.8","opencv-python-headless":"4.13.0.92","opentelemetry-api":"1.40.0","opentelemetry-exporter-otlp":"1.40.0","opentelemetry-exporter-otlp-proto-common":"1.40.0","opentelemetry-exporter-otlp-proto-grpc":"1.40.0","opentelemetry-exporter-otlp-proto-http":"1.40.0","opentelemetry-proto":"1.40.0","opentelemetry-sdk":"1.40.0","opentelemetry-semantic-conventions":"0.61b0","opentelemetry-semantic-conventions-ai":"0.4.15","outlines-core":"0.2.11","packaging":"26.0","partial-json-parser":"0.2.1.1.post7","pillow":"12.1.1","pip":"26.0.1","prometheus-client":"0.24.1","prometheus-fastapi-instrumentator":"7.1.0","propcache":"0.4.1","proto-plus":"1.27.1","protobuf":"6.33.5","psutil":"7.2.2","py-cpuinfo":"9.0.0","pyasn1":"0.6.2","pyasn1-modules":"0.4.2","pybase64":"1.4.3","pycountry":"26.2.16","pycparser":"3.0","pydantic":"2.12.5","pydantic-core":"2.41.5","pydantic-extra-types":"2.11.0","pydantic-settings":"2.13.1","pygments":"2.19.2","pyjwt":"2.11.0","python-dateutil":"2.9.0.post0","python-dotenv":"1.2.2","python-json-logger":"4.0.0","python-multipart":"0.0.22","pyyaml":"6.0.3","pyzmq":"27.1.0","quack-kernels":"0.3.2","ray":"2.54.0","redis":"7.3.0","referencing":"0.37.0","regex":"2026.2.28","requests":"2.32.5","rich":"14.3.3","rich-toolkit":"0.19.7","rignore":"0.7.6","rpds-py":"0.30.0","rsa":"4.9.1","runai-model-streamer":"0.15.7","runai-model-streamer-gcs":"0.15.7","runai-model-streamer-s3":"0.15.7","s3transfer":"0.16.0","safetensors":"0.7.0","sentencepiece":"0.2.1","sentry-sdk":"2.54.0","setproctitle":"1.3.7","setuptools":"80.10.2","setuptools-scm":"9.2.2","shellingham":"1.5.4","six":"1.17.0","sniffio":"1.3.1","sortedcontainers":"2.4.0","sse-starlette":"3.3.2","starlette":"0.52.1","supervisor":"4.3.0","sympy":"1.14.0","synaptic-tuner":"1.1.0","synchronicity":"0.12.5","tabulate":"0.10.0","tiktoken":"0.12.0","timm":"1.0.25","tokenizers":"0.22.2","toml":"0.10.2","torch":"2.10.0+cu129","torch-c-dlpack-ext":"0.1.5","torchaudio":"2.10.0+cu129","torchvision":"0.25.0+cu129","tqdm":"4.67.3","transformers":"4.57.6","triton":"3.6.0","typer":"0.24.1","types-certifi":"2021.10.8.3","types-toml":"0.10.8.20260518","typing-extensions":"4.15.0","typing-inspection":"0.4.2","urllib3":"2.6.3","uv":"0.10.9","uvicorn":"0.41.0","uvloop":"0.22.1","vllm":"0.17.1","watchfiles":"1.1.1","websockets":"16.0","xgrammar":"0.1.29","yarl":"1.23.0","zipp":"3.23.0"},"python":{"executable":"/opt/synaptic-inference/bin/python","executable_sha256":"185be5c01624944b5c422a8d1e4995a7df010036a99c9f9d84a80a09ae1bd72d","implementation":"cpython","version":"3.12.13"},"schema_version":"synaptic-modal-inference-runtime-lock/v1","sdk_version":"1.5.4","source_inventory":[{"path":"Evaluator/__init__.py","sha256":"d1b5aea793a95c47618d4b2469a8bae0b14fd11f81501c1d71546469a39a5cbd","size_bytes":3061},{"path":"Evaluator/base_client.py","sha256":"ac7e77909bdf161a4fd9b1ad2e1e2cfe085ec31f36baa6b19f8976f7b2358cbd","size_bytes":19362},{"path":"Evaluator/config.py","sha256":"94fbe8e198d065141ff9ad002b0a6e26e56d865fc1e3a3fb23479a8e69d09303","size_bytes":21956},{"path":"Evaluator/enums.py","sha256":"2ced9426d860e9ec410e376d9cf853d75e8e111e78dd0bee31e25bee295e4854","size_bytes":3371},{"path":"Evaluator/openai_compat_client.py","sha256":"6ac6211185d2c57d3b31022d31478e3253046fd6cb4de1903d8d0b209bad3a79","size_bytes":5174},{"path":"Evaluator/protocols.py","sha256":"239fe049b5fe4c9e9861aede487feac1f06a7642891378a1ccfba444072ad706","size_bytes":4560},{"path":"Evaluator/verified_vllm_chat.py","sha256":"f130e47d661aa77d68c104084fafe9e5df3af88684ef5a3139f4d53ff2e32465","size_bytes":5916},{"path":"Evaluator/vllm_client.py","sha256":"4e88d2055dc1b32cc22a1393cdf038ccd76d91d915a8be0774b0a8ec87bd4d1e","size_bytes":1995},{"path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"path":"synaptic_tuner/api/v1/planning.py","sha256":"e715b105a513fa56cfa510416e725471f8f3100cb34257be74bef3e40ca78460","size_bytes":9569},{"path":"synaptic_tuner/api/v1/providers.py","sha256":"eca5efd5a6dee453543ae338ffddf9451d34ce1743e4ab4bde024b47be571ec0","size_bytes":4940},{"path":"synaptic_tuner/api/v1/results.py","sha256":"65e530325c92c7afa18e3ca5f2ad9a11baf7d590221f08c33badfbf2d5f2c7b2","size_bytes":2236},{"path":"synaptic_tuner/api/v1/runs_facade.py","sha256":"bf6155726c99cfc5574385009e20f4f65b69a9ad4b96e33cf7065263d1d4c455","size_bytes":23572},{"path":"synaptic_tuner/api/v1/training_facade.py","sha256":"bf094d0e152f5861bf8a8ff51962e0700123d9b43ed91d914b9e78bbea6facd9","size_bytes":11118},{"path":"synaptic_tuner/api/v1/training_input.py","sha256":"209326325f8796ab7d038878e92def2c5521d7d98b7b9cc4dd1766d2e4f68c9d","size_bytes":38650},{"path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"path":"tuner/cloud/runtime_layout.py","sha256":"2a4cd1f5fc63d95bb3f0b5bc4f44e616bb0308d9509aceb198943953746ff705","size_bytes":9435},{"path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"path":"tuner/execution/_effect_executor.py","sha256":"7aa3c523d971486417fe735c62a5cb3fe7e0698caa514d0da047293ad6b3e9cd","size_bytes":500},{"path":"tuner/execution/broker.py","sha256":"c504bd734b41ef9785fe5fa591966bb0b0bce3b43dcce0695b4ee0c142ca3d1e","size_bytes":6305},{"path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"path":"tuner/execution/coordinator_v1/__init__.py","sha256":"453718a1827cfb23e7ca0c908d3d256f62ca3e3fff8f75ed8a785f8336b17bbe","size_bytes":66},{"path":"tuner/execution/coordinator_v1/coordinator.py","sha256":"9bc8ecb60d472ad0b3c0d3192d8c73d88eb28dc43af55677fbdd4b3673b71ef9","size_bytes":47018},{"path":"tuner/execution/coordinator_v1/cursors.py","sha256":"5f7c899064dd26959aeed6b328b236b01f7ff549e2d29348bd45dc0480ede26b","size_bytes":9088},{"path":"tuner/execution/coordinator_v1/foundation.py","sha256":"4433c390075544df5516a5e213af4b4b697485ff7627f5a6dcace21a98029f94","size_bytes":12972},{"path":"tuner/execution/coordinator_v1/model.py","sha256":"1d2150d588f6b84f1b58913df398365a9491d6486a8a871bb2a22fa98cafc722","size_bytes":75837},{"path":"tuner/execution/coordinator_v1/ports.py","sha256":"7a028c32c3a3eab905bbc202248b4bf39411cd22e73fddacf5099b7d15dc3832","size_bytes":9343},{"path":"tuner/execution/coordinator_v1/state_machine.py","sha256":"e5e63db3a85fb9fc5b1d7bb5444691f88f308cfb0428842c0801b06d6f2fbb3a","size_bytes":50176},{"path":"tuner/execution/coordinator_v1/stores.py","sha256":"3a875f9e9d7d7d9899d5deba2f4d68cfada12bea499d5897592c9383ba2b16dc","size_bytes":39045},{"path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"path":"tuner/execution/foundation_v2/__init__.py","sha256":"1651bf897879603a83e7824f60d369d52ac5d0308c96f988c24465f33112cd77","size_bytes":77},{"path":"tuner/execution/foundation_v2/authority.py","sha256":"a0eb5b3537a59c7b4d33e9b8617abd072376545d808f1989f562704dfa1ad024","size_bytes":9958},{"path":"tuner/execution/foundation_v2/canonical.py","sha256":"80620a6088d137bfacb210e000c406632bf569de66350628fb0e0aee612babfb","size_bytes":4047},{"path":"tuner/execution/foundation_v2/commands.py","sha256":"0fff7f05401094a60a390514622bbaa5649a8fb1b5f85f938e856192dac34e5c","size_bytes":8891},{"path":"tuner/execution/foundation_v2/executors.py","sha256":"27a428368c7aa4b9b502ae8028de03b48b817ea9d237a08dd112e7a6afc20454","size_bytes":5331},{"path":"tuner/execution/foundation_v2/identities.py","sha256":"720ae3938627f075c181eb5a125d118df87c9d6f7f24869107c74df9fbc44388","size_bytes":2794},{"path":"tuner/execution/foundation_v2/observations.py","sha256":"3642e4fd35478be5cb9a41df8f758ec4581d9f341f470bad1a4a2fbf8093636c","size_bytes":2566},{"path":"tuner/execution/foundation_v2/operations.py","sha256":"0f3c12d916a71a1fe20017b5ff7e99676c0acbf8bae7fac903d629c1aa3a9e6d","size_bytes":2259},{"path":"tuner/execution/foundation_v2/preparation.py","sha256":"09883f2455e521a7baa9b30fbb81518aa33ab2845392009d87cb54606ff80f1d","size_bytes":3575},{"path":"tuner/execution/foundation_v2/receipts.py","sha256":"c327476257236209ea1e29258fa55d90859bb262d27c132870d21399dfb5f626","size_bytes":15339},{"path":"tuner/execution/foundation_v2/references.py","sha256":"53d8e2070395a4c7f5c5a9361532f7a78de714003c760b55a23366394561b37f","size_bytes":3336},{"path":"tuner/execution/foundation_v2/repository.py","sha256":"d311b21fe8f2ed4b85d50ffa55fca370fe6452841d7a0189c5d56b841f69ef7f","size_bytes":57985},{"path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"path":"tuner/execution/operation.py","sha256":"9f86c585fc7ac3f7b2689a4701d598e32a382c3d1e0b87a7732ef35235568504","size_bytes":13702},{"path":"tuner/execution/providers/__init__.py","sha256":"8848731f5d1877c77ea8fcb66a82b85196aeafc06ba0e7463eca291ec1a5f41d","size_bytes":252},{"path":"tuner/execution/providers/contracts.py","sha256":"a21f22c8d373037d6426cd94dda48d7252438a08e115120188980dcaa34d6361","size_bytes":11506},{"path":"tuner/execution/providers/modal/__init__.py","sha256":"0f4e16bed9b4d05c59a5eef1d0ddc061720d5817a68a2bbe1073b20a19c27319","size_bytes":104},{"path":"tuner/execution/providers/modal/binding.py","sha256":"2d2acb7685798f487988b7690c1dbaf6ddc49e34ed8cd3600add438e17fc9ad5","size_bytes":1810},{"path":"tuner/execution/providers/modal/config.py","sha256":"bfa32ed26f3b8ece5e89e1a3524cb81e85ae3240e72ce93430ffb82dbb02f525","size_bytes":20788},{"path":"tuner/execution/providers/modal/contracts.py","sha256":"249e75a9ad13b93c65ac95cb100dc4fe24ed4c59674770832af3d5a9c4112dd2","size_bytes":6449},{"path":"tuner/execution/providers/modal/control.py","sha256":"19e05126196be5009c2cedac36a39008e82ca0bd17c9ec461f68f6ef26ece750","size_bytes":10359},{"path":"tuner/execution/providers/modal/coordinator_adapter.py","sha256":"619ba5e4e10f1a26ad4c154490da9328f9863581df02bf768e48e045ba46e413","size_bytes":13178},{"path":"tuner/execution/providers/modal/coordinator_binding.py","sha256":"e9267c37dce7273ffcad5f1db5332e205ba5282a9fbad4a647581e23c08b821e","size_bytes":4125},{"path":"tuner/execution/providers/modal/coordinator_bundle.py","sha256":"5d8a10d2c4a0dce83174eeca202c377730820ced6945be6f7717e3a0cfd91eb0","size_bytes":31316},{"path":"tuner/execution/providers/modal/coordinator_dispatch.py","sha256":"8e5bc475c6b7d15345e22e1f8cebdabcffab5a01ffefb7d4acbfb073be02b49f","size_bytes":10571},{"path":"tuner/execution/providers/modal/coordinator_launch.py","sha256":"5975bee2d73dd09e02008ede368ebc8f03325815e8bcfd4c093736214d5f0956","size_bytes":10982},{"path":"tuner/execution/providers/modal/coordinator_preflight.py","sha256":"f452e906d37fa9dba932e9df46f829447c95ec276191e34ec3e46831c89da875","size_bytes":13746},{"path":"tuner/execution/providers/modal/coordinator_reader.py","sha256":"92a1c21725587a88435d30b6b687f63711a12ad4c77345f0776521d6503103f4","size_bytes":28524},{"path":"tuner/execution/providers/modal/coordinator_staging.py","sha256":"de47b688e859ab96a612c01b1c7869d814e40b83ae81dcd7f2b1132e18c794f0","size_bytes":20880},{"path":"tuner/execution/providers/modal/coordinator_submit_preparation.py","sha256":"28814779c7f086016ae3ff617e0f1d4cbd8b52f518d3d889a905ea03d70456ec","size_bytes":5213},{"path":"tuner/execution/providers/modal/coordinator_wire.py","sha256":"07d46b92b33e30cb45bb6b1139ce4f80afdb09e1a17a3f3c2526ddcfae6983c9","size_bytes":12998},{"path":"tuner/execution/providers/modal/deployment_identity.py","sha256":"1b7007e5a9c11abdfe0ad4f106f742faa0ee556921b4017d3b48cb772bd0da4f","size_bytes":1196},{"path":"tuner/execution/providers/modal/deployment_v1.py","sha256":"579ace8adbddfe30e09d5a39df9926a54b111a139c704fc7efc7cebf533acf94","size_bytes":3628},{"path":"tuner/execution/providers/modal/facade.py","sha256":"55b89b3d0faff6238eeead068d6963d26f27749d725d2f2b8ae32449b348e32d","size_bytes":17704},{"path":"tuner/execution/providers/modal/inference-dependencies.lock","sha256":"ca4e3f0d7fa06533ad1127d84b539279791e37f07dc62cc81fbf0edd3a59de88","size_bytes":8458},{"path":"tuner/execution/providers/modal/inference-worker-closure.json","sha256":"cf21d2b602504a7ccccd907bf8358d485501dc6798eae8a0779b70de5c305fd6","size_bytes":18118},{"path":"tuner/execution/providers/modal/inference_artifacts.py","sha256":"5816f0c5662de8edad752554ba149d9527ced40ebdceeccc993c136ab099bff3","size_bytes":14973},{"path":"tuner/execution/providers/modal/inference_binding.py","sha256":"f2e8c4f1d8fc1f5e70702c88cbfff044a1f4f569e06b9a31c33a0cb9aaaf5894","size_bytes":23363},{"path":"tuner/execution/providers/modal/inference_bootstrap.py","sha256":"db2f09c1dfb2aee667b46fbb6566be8b2922d8e762896e5577d5c5078ad7d122","size_bytes":12546},{"path":"tuner/execution/providers/modal/inference_channel.py","sha256":"23492b4c7f48d86d23fb21ffd3902ac3f48bae381c30ea4759c803577911592e","size_bytes":13438},{"path":"tuner/execution/providers/modal/inference_commands.py","sha256":"1fa2dd667cb3b33e012d37b0688bfe08fc7a7cec52230a202f1f9ddda70110c5","size_bytes":4977},{"path":"tuner/execution/providers/modal/inference_entrypoint.py","sha256":"6ec8bdcbbf6d87fb5fc02a2c42d2c0f344ea965ccc4c0d94a630055c6e779991","size_bytes":10873},{"path":"tuner/execution/providers/modal/inference_model.py","sha256":"c08e1a35bb71a11c3466aedfbd8d3de87cf32d6cbd85820c3dcde2fefd14fd65","size_bytes":5935},{"path":"tuner/execution/providers/modal/inference_preparation.py","sha256":"43a1c883a8e5b2f25364e3a20f59d92a852af15fda0c9d506c213c0bf185fc32","size_bytes":49436},{"path":"tuner/execution/providers/modal/inference_runtime.py","sha256":"f8ba0d5744f90ca95e1fea3e5faf293e153a91047ffb09596f0305e1339d56ee","size_bytes":19283},{"path":"tuner/execution/providers/modal/inference_wire.py","sha256":"0492c10c6a384596bba04b3b409dc34f1f1a4afa88b7321d0d1a6fac5e29a1fb","size_bytes":20150},{"path":"tuner/execution/providers/modal/inference_worker.py","sha256":"cfb27e938c60757ef08c80b918fb522012c84918e7a0cd41a63de2cb2486cb87","size_bytes":15657},{"path":"tuner/execution/providers/modal/inference_workload.py","sha256":"4eaf399485e76829a98f57907c012658ce41d02745978827f21262ce2be94ff8","size_bytes":20421},{"path":"tuner/execution/providers/modal/manifest.py","sha256":"fc2ab4dcc46d136a681bf124d7a9fc3e5deedeeb3641d798fd207cdb62bb506f","size_bytes":8579},{"path":"tuner/execution/providers/modal/model_snapshot.py","sha256":"9d4762b821b2944a8a62932d0aec105c2f6920d64bf4ceed6907e5458799e337","size_bytes":14166},{"path":"tuner/execution/providers/modal/mounted_io.py","sha256":"e49da2e749d3be1ca19fdc7f581b29fb6d1e748a86fffcf0fde2be5cdc1a0d5b","size_bytes":13076},{"path":"tuner/execution/providers/modal/prepared_input.py","sha256":"c16cef5cfcc73ab9a7826977b26e18912ac46409d91b9eae165893ed265896ac","size_bytes":8597},{"path":"tuner/execution/providers/modal/resolution.py","sha256":"5c7268aa644103ae0712b03ed55918527d6e2ed6f4120ef169866b0ffd1cbf6f","size_bytes":30179},{"path":"tuner/execution/providers/modal/runtime.py","sha256":"ada13733021a6dab19331da80076df630055abc5bddc192a34a5cda65ac718c4","size_bytes":14810},{"path":"tuner/execution/providers/modal/worker_ports.py","sha256":"adf334368dbb75e40177078cd32aedc445cd9dde8b5ece2401319d472dbccdb0","size_bytes":2928},{"path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"path":"tuner/inference/__init__.py","sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","size_bytes":0},{"path":"tuner/inference/chat_session.py","sha256":"b95a829f834406a0a0c152f9af713225fe205a9225ad7b0f959b1ba801dbcac2","size_bytes":22355},{"path":"tuner/inference/owned_process.py","sha256":"f4cbb4e1c2305615b2c127e1103f60b1e17e955fe20a2a847b1d88b7ba8c4c98","size_bytes":8832},{"path":"tuner/inference/retrieved_model.py","sha256":"4a8cf2b2cbe4f03bf7c4d240c5e59a6ec6d20826c686b3a06c0bc1642fa234d8","size_bytes":31703},{"path":"tuner/inference/run_chat.py","sha256":"9263d325224bee4b9a0e7c16045e30766d9a2e2d2b550f0508040161687d3cfc","size_bytes":6228},{"path":"tuner/inference/serving_target.py","sha256":"50ec8054d1d6d08724d7a29183ca54a04d9c915d2c04274bdac6834764bdb800","size_bytes":12227},{"path":"tuner/inference/vllm_runtime.py","sha256":"fafbcfdeee78143a2760e5fbf29621897e9be014814eefb69f1167b188c296c1","size_bytes":26019},{"path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"path":"tuner/runtime/dispatch.py","sha256":"4c41faf87d7e620bdc281490bf3725550b83dacca8b15ea47bab253f669cdd8e","size_bytes":35463},{"path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"path":"tuner/runtime/verification.py","sha256":"4c2314b20cbf7937a3f24aa90676000b9e21f0ab6701b2c1225a53e5596bf4a0","size_bytes":53856},{"path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"path":"tuner/training/contracts.py","sha256":"91c4d66d9810cc0471981c3d4ef4b54feaed5b92eb41a468f31201f8111303b5","size_bytes":21504},{"path":"tuner/training/coordinator_material.py","sha256":"c563aa3b1fbd940ddd3b8e56e4f33de70675cb831e938175f40debc0645fd9c7","size_bytes":12950},{"path":"tuner/training/grpo_compilation.py","sha256":"c6268486375bbc16f1888bb43d07b93a8de280d000a774914d59d7fd05ca5cc0","size_bytes":8998},{"path":"tuner/training/methods/__init__.py","sha256":"99db689246273707ae3df5087859ebb510e84b664f9f173c0d5827abc3e6a3ea","size_bytes":361},{"path":"tuner/training/methods/sft.py","sha256":"b501f9c214dec4b0b1550543281c25aba8dbb102604df253c641c85f7cae587b","size_bytes":9997},{"path":"tuner/training/packaged_compilation.py","sha256":"477caede6902fc587af680889b93fd9e0e42493d1a6653f3cd7413a9c7cadde8","size_bytes":9413},{"path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722},{"path":"tuner/training/post_training.py","sha256":"ba8c2ea3fe49a647d502c6a874a44658cc2e575d4cd5729966102b0ebb973a88","size_bytes":7971},{"path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"worker_closure_manifest_path":"tuner/execution/providers/modal/inference-worker-closure.json"} +{"base_registry_reference":"docker.io/vllm/vllm-openai@sha256:116aa00ee0b68855616a56e1d7e1ae937e591a8bd6969ee45cbcedb246ddf355","dependency_lock_path":"tuner/execution/providers/modal/inference-dependencies.lock","distributions":{"accelerate":"1.13.0","aiofile":"3.9.0","aiofiles":"25.1.0","aiohappyeyeballs":"2.6.1","aiohttp":"3.13.3","aiosignal":"1.4.0","annotated-doc":"0.0.4","annotated-types":"0.7.0","anthropic":"0.84.0","anyio":"4.12.1","apache-tvm-ffi":"0.1.9","astor":"0.8.1","attrs":"25.4.0","awscrt":"0.31.3","bitsandbytes":"0.49.2","blake3":"1.0.8","boto3":"1.42.65","botocore":"1.42.65","cachetools":"7.0.5","caio":"0.9.25","cbor2":"5.8.0","certifi":"2026.2.25","cffi":"2.0.0","charset-normalizer":"3.4.5","click":"8.3.1","cloudpickle":"3.1.2","compressed-tensors":"0.13.0","cryptography":"46.0.5","cuda-bindings":"12.9.4","cuda-pathfinder":"1.4.2","cuda-python":"12.9.4","cufile-python":"0.2.0","cupy-cuda12x":"14.0.1","deep-ep":"1.2.1+73b6ea4","deep-gemm":"2.3.0+477618c","depyf":"0.20.0","dill":"0.4.1","diskcache":"5.6.3","distro":"1.9.0","dnspython":"2.8.0","docstring-parser":"0.17.0","einops":"0.8.2","email-validator":"2.3.0","fastapi":"0.135.1","fastapi-cli":"0.0.24","fastapi-cloud-cli":"0.14.1","fastar":"0.8.0","filelock":"3.25.1","flashinfer-cubin":"0.6.4","flashinfer-jit-cache":"0.6.4+cu129","flashinfer-python":"0.6.4","frozenlist":"1.8.0","fsspec":"2026.2.0","gguf":"0.18.0","google-api-core":"2.30.0","google-auth":"2.49.0","google-cloud-core":"2.5.0","google-cloud-storage":"3.9.0","google-crc32c":"1.8.0","google-resumable-media":"2.8.0","googleapis-common-protos":"1.73.0","grpcio":"1.78.0","grpcio-reflection":"1.78.0","grpclib":"0.4.9","h11":"0.16.0","h2":"4.4.1","hf-transfer":"0.1.9","hf-xet":"1.3.2","hpack":"4.2.0","httpcore":"1.0.9","httptools":"0.7.1","httpx":"0.28.1","httpx-sse":"0.4.3","huggingface-hub":"0.36.2","humanize":"4.15.0","hyperframe":"6.1.0","idna":"3.11","ijson":"3.5.0","importlib-metadata":"8.7.1","interegular":"0.3.3","jinja2":"3.1.6","jiter":"0.13.0","jmespath":"1.1.0","jsonschema":"4.26.0","jsonschema-specifications":"2025.9.1","kaldi-native-fbank":"1.22.3","lark":"1.2.2","llguidance":"1.3.0","llvmlite":"0.44.0","lm-format-enforcer":"0.11.3","lmcache":"0.4.0","loguru":"0.7.3","markdown-it-py":"4.0.0","markupsafe":"3.0.3","mcp":"1.26.0","mdurl":"0.1.2","mistral-common":"1.9.1","modal":"1.5.4","model-hosting-container-standards":"0.1.13","modelscope":"1.34.0","mooncake-transfer-engine":"0.3.9","mpmath":"1.3.0","msgpack":"1.1.2","msgspec":"0.20.0","multidict":"6.7.1","networkx":"3.6.1","ninja":"1.13.0","nixl":"0.9.0","nixl-cu12":"0.10.1","numba":"0.61.2","numpy":"2.2.6","nvidia-cublas-cu12":"12.9.1.4","nvidia-cuda-cupti-cu12":"12.9.79","nvidia-cuda-nvrtc-cu12":"12.9.86","nvidia-cuda-runtime-cu12":"12.9.79","nvidia-cudnn-cu12":"9.10.2.21","nvidia-cudnn-frontend":"1.18.0","nvidia-cufft-cu12":"11.4.1.4","nvidia-cufile-cu12":"1.14.1.1","nvidia-curand-cu12":"10.3.10.19","nvidia-cusolver-cu12":"11.7.5.82","nvidia-cusparse-cu12":"12.5.10.65","nvidia-cusparselt-cu12":"0.7.1","nvidia-cutlass-dsl":"4.4.1","nvidia-cutlass-dsl-libs-base":"4.4.1","nvidia-ml-py":"13.590.48","nvidia-nccl-cu12":"2.27.5","nvidia-nvjitlink-cu12":"12.9.86","nvidia-nvshmem-cu12":"3.4.5","nvidia-nvtx-cu12":"12.9.79","nvtx":"0.2.14","openai":"2.24.0","openai-harmony":"0.0.8","opencv-python-headless":"4.13.0.92","opentelemetry-api":"1.40.0","opentelemetry-exporter-otlp":"1.40.0","opentelemetry-exporter-otlp-proto-common":"1.40.0","opentelemetry-exporter-otlp-proto-grpc":"1.40.0","opentelemetry-exporter-otlp-proto-http":"1.40.0","opentelemetry-proto":"1.40.0","opentelemetry-sdk":"1.40.0","opentelemetry-semantic-conventions":"0.61b0","opentelemetry-semantic-conventions-ai":"0.4.15","outlines-core":"0.2.11","packaging":"26.0","partial-json-parser":"0.2.1.1.post7","pillow":"12.1.1","pip":"26.0.1","prometheus-client":"0.24.1","prometheus-fastapi-instrumentator":"7.1.0","propcache":"0.4.1","proto-plus":"1.27.1","protobuf":"6.33.5","psutil":"7.2.2","py-cpuinfo":"9.0.0","pyasn1":"0.6.2","pyasn1-modules":"0.4.2","pybase64":"1.4.3","pycountry":"26.2.16","pycparser":"3.0","pydantic":"2.12.5","pydantic-core":"2.41.5","pydantic-extra-types":"2.11.0","pydantic-settings":"2.13.1","pygments":"2.19.2","pyjwt":"2.11.0","python-dateutil":"2.9.0.post0","python-dotenv":"1.2.2","python-json-logger":"4.0.0","python-multipart":"0.0.22","pyyaml":"6.0.3","pyzmq":"27.1.0","quack-kernels":"0.3.2","ray":"2.54.0","redis":"7.3.0","referencing":"0.37.0","regex":"2026.2.28","requests":"2.32.5","rich":"14.3.3","rich-toolkit":"0.19.7","rignore":"0.7.6","rpds-py":"0.30.0","rsa":"4.9.1","runai-model-streamer":"0.15.7","runai-model-streamer-gcs":"0.15.7","runai-model-streamer-s3":"0.15.7","s3transfer":"0.16.0","safetensors":"0.7.0","sentencepiece":"0.2.1","sentry-sdk":"2.54.0","setproctitle":"1.3.7","setuptools":"80.10.2","setuptools-scm":"9.2.2","shellingham":"1.5.4","six":"1.17.0","sniffio":"1.3.1","sortedcontainers":"2.4.0","sse-starlette":"3.3.2","starlette":"0.52.1","supervisor":"4.3.0","sympy":"1.14.0","synaptic-tuner":"1.1.0","synchronicity":"0.12.5","tabulate":"0.10.0","tiktoken":"0.12.0","timm":"1.0.25","tokenizers":"0.22.2","toml":"0.10.2","torch":"2.10.0+cu129","torch-c-dlpack-ext":"0.1.5","torchaudio":"2.10.0+cu129","torchvision":"0.25.0+cu129","tqdm":"4.67.3","transformers":"4.57.6","triton":"3.6.0","typer":"0.24.1","types-certifi":"2021.10.8.3","types-toml":"0.10.8.20260518","typing-extensions":"4.15.0","typing-inspection":"0.4.2","urllib3":"2.6.3","uv":"0.10.9","uvicorn":"0.41.0","uvloop":"0.22.1","vllm":"0.17.1","watchfiles":"1.1.1","websockets":"16.0","xgrammar":"0.1.29","yarl":"1.23.0","zipp":"3.23.0"},"python":{"executable":"/opt/synaptic-inference/bin/python","executable_sha256":"185be5c01624944b5c422a8d1e4995a7df010036a99c9f9d84a80a09ae1bd72d","implementation":"cpython","version":"3.12.13"},"schema_version":"synaptic-modal-inference-runtime-lock/v1","sdk_version":"1.5.4","source_inventory":[{"path":"Evaluator/__init__.py","sha256":"d1b5aea793a95c47618d4b2469a8bae0b14fd11f81501c1d71546469a39a5cbd","size_bytes":3061},{"path":"Evaluator/base_client.py","sha256":"ac7e77909bdf161a4fd9b1ad2e1e2cfe085ec31f36baa6b19f8976f7b2358cbd","size_bytes":19362},{"path":"Evaluator/config.py","sha256":"94fbe8e198d065141ff9ad002b0a6e26e56d865fc1e3a3fb23479a8e69d09303","size_bytes":21956},{"path":"Evaluator/enums.py","sha256":"2ced9426d860e9ec410e376d9cf853d75e8e111e78dd0bee31e25bee295e4854","size_bytes":3371},{"path":"Evaluator/openai_compat_client.py","sha256":"6ac6211185d2c57d3b31022d31478e3253046fd6cb4de1903d8d0b209bad3a79","size_bytes":5174},{"path":"Evaluator/protocols.py","sha256":"239fe049b5fe4c9e9861aede487feac1f06a7642891378a1ccfba444072ad706","size_bytes":4560},{"path":"Evaluator/verified_vllm_chat.py","sha256":"f130e47d661aa77d68c104084fafe9e5df3af88684ef5a3139f4d53ff2e32465","size_bytes":5916},{"path":"Evaluator/vllm_client.py","sha256":"4e88d2055dc1b32cc22a1393cdf038ccd76d91d915a8be0774b0a8ec87bd4d1e","size_bytes":1995},{"path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"path":"synaptic_tuner/api/v1/planning.py","sha256":"e715b105a513fa56cfa510416e725471f8f3100cb34257be74bef3e40ca78460","size_bytes":9569},{"path":"synaptic_tuner/api/v1/providers.py","sha256":"eca5efd5a6dee453543ae338ffddf9451d34ce1743e4ab4bde024b47be571ec0","size_bytes":4940},{"path":"synaptic_tuner/api/v1/results.py","sha256":"65e530325c92c7afa18e3ca5f2ad9a11baf7d590221f08c33badfbf2d5f2c7b2","size_bytes":2236},{"path":"synaptic_tuner/api/v1/runs_facade.py","sha256":"bf6155726c99cfc5574385009e20f4f65b69a9ad4b96e33cf7065263d1d4c455","size_bytes":23572},{"path":"synaptic_tuner/api/v1/training_facade.py","sha256":"bf094d0e152f5861bf8a8ff51962e0700123d9b43ed91d914b9e78bbea6facd9","size_bytes":11118},{"path":"synaptic_tuner/api/v1/training_input.py","sha256":"209326325f8796ab7d038878e92def2c5521d7d98b7b9cc4dd1766d2e4f68c9d","size_bytes":38650},{"path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"path":"tuner/cloud/runtime_layout.py","sha256":"2a4cd1f5fc63d95bb3f0b5bc4f44e616bb0308d9509aceb198943953746ff705","size_bytes":9435},{"path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"path":"tuner/execution/_effect_executor.py","sha256":"7aa3c523d971486417fe735c62a5cb3fe7e0698caa514d0da047293ad6b3e9cd","size_bytes":500},{"path":"tuner/execution/broker.py","sha256":"c504bd734b41ef9785fe5fa591966bb0b0bce3b43dcce0695b4ee0c142ca3d1e","size_bytes":6305},{"path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"path":"tuner/execution/coordinator_v1/__init__.py","sha256":"453718a1827cfb23e7ca0c908d3d256f62ca3e3fff8f75ed8a785f8336b17bbe","size_bytes":66},{"path":"tuner/execution/coordinator_v1/coordinator.py","sha256":"9bc8ecb60d472ad0b3c0d3192d8c73d88eb28dc43af55677fbdd4b3673b71ef9","size_bytes":47018},{"path":"tuner/execution/coordinator_v1/cursors.py","sha256":"5f7c899064dd26959aeed6b328b236b01f7ff549e2d29348bd45dc0480ede26b","size_bytes":9088},{"path":"tuner/execution/coordinator_v1/foundation.py","sha256":"4433c390075544df5516a5e213af4b4b697485ff7627f5a6dcace21a98029f94","size_bytes":12972},{"path":"tuner/execution/coordinator_v1/model.py","sha256":"1d2150d588f6b84f1b58913df398365a9491d6486a8a871bb2a22fa98cafc722","size_bytes":75837},{"path":"tuner/execution/coordinator_v1/ports.py","sha256":"7a028c32c3a3eab905bbc202248b4bf39411cd22e73fddacf5099b7d15dc3832","size_bytes":9343},{"path":"tuner/execution/coordinator_v1/state_machine.py","sha256":"e5e63db3a85fb9fc5b1d7bb5444691f88f308cfb0428842c0801b06d6f2fbb3a","size_bytes":50176},{"path":"tuner/execution/coordinator_v1/stores.py","sha256":"3a875f9e9d7d7d9899d5deba2f4d68cfada12bea499d5897592c9383ba2b16dc","size_bytes":39045},{"path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"path":"tuner/execution/foundation_v2/__init__.py","sha256":"1651bf897879603a83e7824f60d369d52ac5d0308c96f988c24465f33112cd77","size_bytes":77},{"path":"tuner/execution/foundation_v2/authority.py","sha256":"a0eb5b3537a59c7b4d33e9b8617abd072376545d808f1989f562704dfa1ad024","size_bytes":9958},{"path":"tuner/execution/foundation_v2/canonical.py","sha256":"80620a6088d137bfacb210e000c406632bf569de66350628fb0e0aee612babfb","size_bytes":4047},{"path":"tuner/execution/foundation_v2/commands.py","sha256":"0fff7f05401094a60a390514622bbaa5649a8fb1b5f85f938e856192dac34e5c","size_bytes":8891},{"path":"tuner/execution/foundation_v2/executors.py","sha256":"27a428368c7aa4b9b502ae8028de03b48b817ea9d237a08dd112e7a6afc20454","size_bytes":5331},{"path":"tuner/execution/foundation_v2/identities.py","sha256":"720ae3938627f075c181eb5a125d118df87c9d6f7f24869107c74df9fbc44388","size_bytes":2794},{"path":"tuner/execution/foundation_v2/observations.py","sha256":"3642e4fd35478be5cb9a41df8f758ec4581d9f341f470bad1a4a2fbf8093636c","size_bytes":2566},{"path":"tuner/execution/foundation_v2/operations.py","sha256":"0f3c12d916a71a1fe20017b5ff7e99676c0acbf8bae7fac903d629c1aa3a9e6d","size_bytes":2259},{"path":"tuner/execution/foundation_v2/preparation.py","sha256":"09883f2455e521a7baa9b30fbb81518aa33ab2845392009d87cb54606ff80f1d","size_bytes":3575},{"path":"tuner/execution/foundation_v2/receipts.py","sha256":"c327476257236209ea1e29258fa55d90859bb262d27c132870d21399dfb5f626","size_bytes":15339},{"path":"tuner/execution/foundation_v2/references.py","sha256":"53d8e2070395a4c7f5c5a9361532f7a78de714003c760b55a23366394561b37f","size_bytes":3336},{"path":"tuner/execution/foundation_v2/repository.py","sha256":"d311b21fe8f2ed4b85d50ffa55fca370fe6452841d7a0189c5d56b841f69ef7f","size_bytes":57985},{"path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"path":"tuner/execution/operation.py","sha256":"9f86c585fc7ac3f7b2689a4701d598e32a382c3d1e0b87a7732ef35235568504","size_bytes":13702},{"path":"tuner/execution/providers/__init__.py","sha256":"8848731f5d1877c77ea8fcb66a82b85196aeafc06ba0e7463eca291ec1a5f41d","size_bytes":252},{"path":"tuner/execution/providers/contracts.py","sha256":"a21f22c8d373037d6426cd94dda48d7252438a08e115120188980dcaa34d6361","size_bytes":11506},{"path":"tuner/execution/providers/modal/__init__.py","sha256":"0f4e16bed9b4d05c59a5eef1d0ddc061720d5817a68a2bbe1073b20a19c27319","size_bytes":104},{"path":"tuner/execution/providers/modal/binding.py","sha256":"2d2acb7685798f487988b7690c1dbaf6ddc49e34ed8cd3600add438e17fc9ad5","size_bytes":1810},{"path":"tuner/execution/providers/modal/config.py","sha256":"bfa32ed26f3b8ece5e89e1a3524cb81e85ae3240e72ce93430ffb82dbb02f525","size_bytes":20788},{"path":"tuner/execution/providers/modal/contracts.py","sha256":"249e75a9ad13b93c65ac95cb100dc4fe24ed4c59674770832af3d5a9c4112dd2","size_bytes":6449},{"path":"tuner/execution/providers/modal/control.py","sha256":"19e05126196be5009c2cedac36a39008e82ca0bd17c9ec461f68f6ef26ece750","size_bytes":10359},{"path":"tuner/execution/providers/modal/coordinator_adapter.py","sha256":"619ba5e4e10f1a26ad4c154490da9328f9863581df02bf768e48e045ba46e413","size_bytes":13178},{"path":"tuner/execution/providers/modal/coordinator_binding.py","sha256":"e9267c37dce7273ffcad5f1db5332e205ba5282a9fbad4a647581e23c08b821e","size_bytes":4125},{"path":"tuner/execution/providers/modal/coordinator_bundle.py","sha256":"5d8a10d2c4a0dce83174eeca202c377730820ced6945be6f7717e3a0cfd91eb0","size_bytes":31316},{"path":"tuner/execution/providers/modal/coordinator_dispatch.py","sha256":"8e5bc475c6b7d15345e22e1f8cebdabcffab5a01ffefb7d4acbfb073be02b49f","size_bytes":10571},{"path":"tuner/execution/providers/modal/coordinator_launch.py","sha256":"5975bee2d73dd09e02008ede368ebc8f03325815e8bcfd4c093736214d5f0956","size_bytes":10982},{"path":"tuner/execution/providers/modal/coordinator_preflight.py","sha256":"f452e906d37fa9dba932e9df46f829447c95ec276191e34ec3e46831c89da875","size_bytes":13746},{"path":"tuner/execution/providers/modal/coordinator_reader.py","sha256":"92a1c21725587a88435d30b6b687f63711a12ad4c77345f0776521d6503103f4","size_bytes":28524},{"path":"tuner/execution/providers/modal/coordinator_staging.py","sha256":"de47b688e859ab96a612c01b1c7869d814e40b83ae81dcd7f2b1132e18c794f0","size_bytes":20880},{"path":"tuner/execution/providers/modal/coordinator_submit_preparation.py","sha256":"28814779c7f086016ae3ff617e0f1d4cbd8b52f518d3d889a905ea03d70456ec","size_bytes":5213},{"path":"tuner/execution/providers/modal/coordinator_wire.py","sha256":"07d46b92b33e30cb45bb6b1139ce4f80afdb09e1a17a3f3c2526ddcfae6983c9","size_bytes":12998},{"path":"tuner/execution/providers/modal/deployment_identity.py","sha256":"1b7007e5a9c11abdfe0ad4f106f742faa0ee556921b4017d3b48cb772bd0da4f","size_bytes":1196},{"path":"tuner/execution/providers/modal/deployment_v1.py","sha256":"579ace8adbddfe30e09d5a39df9926a54b111a139c704fc7efc7cebf533acf94","size_bytes":3628},{"path":"tuner/execution/providers/modal/facade.py","sha256":"55b89b3d0faff6238eeead068d6963d26f27749d725d2f2b8ae32449b348e32d","size_bytes":17704},{"path":"tuner/execution/providers/modal/inference-dependencies.lock","sha256":"ca4e3f0d7fa06533ad1127d84b539279791e37f07dc62cc81fbf0edd3a59de88","size_bytes":8458},{"path":"tuner/execution/providers/modal/inference-worker-closure.json","sha256":"ba458bb036f9349d558dab45b693899648a11dd69d40b01548bb0b3fa333c6b9","size_bytes":18118},{"path":"tuner/execution/providers/modal/inference_artifacts.py","sha256":"5816f0c5662de8edad752554ba149d9527ced40ebdceeccc993c136ab099bff3","size_bytes":14973},{"path":"tuner/execution/providers/modal/inference_binding.py","sha256":"f2e8c4f1d8fc1f5e70702c88cbfff044a1f4f569e06b9a31c33a0cb9aaaf5894","size_bytes":23363},{"path":"tuner/execution/providers/modal/inference_bootstrap.py","sha256":"db2f09c1dfb2aee667b46fbb6566be8b2922d8e762896e5577d5c5078ad7d122","size_bytes":12546},{"path":"tuner/execution/providers/modal/inference_channel.py","sha256":"23492b4c7f48d86d23fb21ffd3902ac3f48bae381c30ea4759c803577911592e","size_bytes":13438},{"path":"tuner/execution/providers/modal/inference_commands.py","sha256":"1fa2dd667cb3b33e012d37b0688bfe08fc7a7cec52230a202f1f9ddda70110c5","size_bytes":4977},{"path":"tuner/execution/providers/modal/inference_entrypoint.py","sha256":"6ec8bdcbbf6d87fb5fc02a2c42d2c0f344ea965ccc4c0d94a630055c6e779991","size_bytes":10873},{"path":"tuner/execution/providers/modal/inference_model.py","sha256":"c08e1a35bb71a11c3466aedfbd8d3de87cf32d6cbd85820c3dcde2fefd14fd65","size_bytes":5935},{"path":"tuner/execution/providers/modal/inference_preparation.py","sha256":"43a1c883a8e5b2f25364e3a20f59d92a852af15fda0c9d506c213c0bf185fc32","size_bytes":49436},{"path":"tuner/execution/providers/modal/inference_runtime.py","sha256":"f8ba0d5744f90ca95e1fea3e5faf293e153a91047ffb09596f0305e1339d56ee","size_bytes":19283},{"path":"tuner/execution/providers/modal/inference_wire.py","sha256":"0492c10c6a384596bba04b3b409dc34f1f1a4afa88b7321d0d1a6fac5e29a1fb","size_bytes":20150},{"path":"tuner/execution/providers/modal/inference_worker.py","sha256":"cfb27e938c60757ef08c80b918fb522012c84918e7a0cd41a63de2cb2486cb87","size_bytes":15657},{"path":"tuner/execution/providers/modal/inference_workload.py","sha256":"4eaf399485e76829a98f57907c012658ce41d02745978827f21262ce2be94ff8","size_bytes":20421},{"path":"tuner/execution/providers/modal/manifest.py","sha256":"fc2ab4dcc46d136a681bf124d7a9fc3e5deedeeb3641d798fd207cdb62bb506f","size_bytes":8579},{"path":"tuner/execution/providers/modal/model_snapshot.py","sha256":"9d4762b821b2944a8a62932d0aec105c2f6920d64bf4ceed6907e5458799e337","size_bytes":14166},{"path":"tuner/execution/providers/modal/mounted_io.py","sha256":"e49da2e749d3be1ca19fdc7f581b29fb6d1e748a86fffcf0fde2be5cdc1a0d5b","size_bytes":13076},{"path":"tuner/execution/providers/modal/prepared_input.py","sha256":"c16cef5cfcc73ab9a7826977b26e18912ac46409d91b9eae165893ed265896ac","size_bytes":8597},{"path":"tuner/execution/providers/modal/resolution.py","sha256":"5c7268aa644103ae0712b03ed55918527d6e2ed6f4120ef169866b0ffd1cbf6f","size_bytes":30179},{"path":"tuner/execution/providers/modal/runtime.py","sha256":"ada13733021a6dab19331da80076df630055abc5bddc192a34a5cda65ac718c4","size_bytes":14810},{"path":"tuner/execution/providers/modal/worker_ports.py","sha256":"adf334368dbb75e40177078cd32aedc445cd9dde8b5ece2401319d472dbccdb0","size_bytes":2928},{"path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"path":"tuner/inference/__init__.py","sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","size_bytes":0},{"path":"tuner/inference/chat_session.py","sha256":"b95a829f834406a0a0c152f9af713225fe205a9225ad7b0f959b1ba801dbcac2","size_bytes":22355},{"path":"tuner/inference/owned_process.py","sha256":"f4cbb4e1c2305615b2c127e1103f60b1e17e955fe20a2a847b1d88b7ba8c4c98","size_bytes":8832},{"path":"tuner/inference/retrieved_model.py","sha256":"4a8cf2b2cbe4f03bf7c4d240c5e59a6ec6d20826c686b3a06c0bc1642fa234d8","size_bytes":31703},{"path":"tuner/inference/run_chat.py","sha256":"9263d325224bee4b9a0e7c16045e30766d9a2e2d2b550f0508040161687d3cfc","size_bytes":6228},{"path":"tuner/inference/serving_target.py","sha256":"50ec8054d1d6d08724d7a29183ca54a04d9c915d2c04274bdac6834764bdb800","size_bytes":12227},{"path":"tuner/inference/vllm_runtime.py","sha256":"fafbcfdeee78143a2760e5fbf29621897e9be014814eefb69f1167b188c296c1","size_bytes":26019},{"path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"path":"tuner/runtime/dispatch.py","sha256":"4c41faf87d7e620bdc281490bf3725550b83dacca8b15ea47bab253f669cdd8e","size_bytes":35463},{"path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"path":"tuner/runtime/verification.py","sha256":"4c2314b20cbf7937a3f24aa90676000b9e21f0ab6701b2c1225a53e5596bf4a0","size_bytes":53856},{"path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"path":"tuner/training/contracts.py","sha256":"ebe52af8ba1c02309d39317694bc04547a6491d110c60229282517dbdab5a417","size_bytes":21524},{"path":"tuner/training/coordinator_material.py","sha256":"c563aa3b1fbd940ddd3b8e56e4f33de70675cb831e938175f40debc0645fd9c7","size_bytes":12950},{"path":"tuner/training/grpo_compilation.py","sha256":"c6268486375bbc16f1888bb43d07b93a8de280d000a774914d59d7fd05ca5cc0","size_bytes":8998},{"path":"tuner/training/methods/__init__.py","sha256":"99db689246273707ae3df5087859ebb510e84b664f9f173c0d5827abc3e6a3ea","size_bytes":361},{"path":"tuner/training/methods/sft.py","sha256":"b501f9c214dec4b0b1550543281c25aba8dbb102604df253c641c85f7cae587b","size_bytes":9997},{"path":"tuner/training/packaged_compilation.py","sha256":"477caede6902fc587af680889b93fd9e0e42493d1a6653f3cd7413a9c7cadde8","size_bytes":9413},{"path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722},{"path":"tuner/training/post_training.py","sha256":"ba8c2ea3fe49a647d502c6a874a44658cc2e575d4cd5729966102b0ebb973a88","size_bytes":7971},{"path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"worker_closure_manifest_path":"tuner/execution/providers/modal/inference-worker-closure.json"} diff --git a/tuner/execution/providers/modal/inference-worker-closure.json b/tuner/execution/providers/modal/inference-worker-closure.json index 65c5551a..c650be43 100644 --- a/tuner/execution/providers/modal/inference-worker-closure.json +++ b/tuner/execution/providers/modal/inference-worker-closure.json @@ -1 +1 @@ -{"closure_digest":"11b8b9df877b6dbb3a50268f7782e8ed0f6b291e85a5b4981ff0a2f8edf47175","entrypoint":"tuner/execution/providers/modal/inference_bootstrap.py","member_count":124,"members":[{"path":"Evaluator/__init__.py","sha256":"d1b5aea793a95c47618d4b2469a8bae0b14fd11f81501c1d71546469a39a5cbd","size_bytes":3061},{"path":"Evaluator/base_client.py","sha256":"ac7e77909bdf161a4fd9b1ad2e1e2cfe085ec31f36baa6b19f8976f7b2358cbd","size_bytes":19362},{"path":"Evaluator/config.py","sha256":"94fbe8e198d065141ff9ad002b0a6e26e56d865fc1e3a3fb23479a8e69d09303","size_bytes":21956},{"path":"Evaluator/enums.py","sha256":"2ced9426d860e9ec410e376d9cf853d75e8e111e78dd0bee31e25bee295e4854","size_bytes":3371},{"path":"Evaluator/openai_compat_client.py","sha256":"6ac6211185d2c57d3b31022d31478e3253046fd6cb4de1903d8d0b209bad3a79","size_bytes":5174},{"path":"Evaluator/protocols.py","sha256":"239fe049b5fe4c9e9861aede487feac1f06a7642891378a1ccfba444072ad706","size_bytes":4560},{"path":"Evaluator/verified_vllm_chat.py","sha256":"f130e47d661aa77d68c104084fafe9e5df3af88684ef5a3139f4d53ff2e32465","size_bytes":5916},{"path":"Evaluator/vllm_client.py","sha256":"4e88d2055dc1b32cc22a1393cdf038ccd76d91d915a8be0774b0a8ec87bd4d1e","size_bytes":1995},{"path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"path":"synaptic_tuner/api/v1/planning.py","sha256":"e715b105a513fa56cfa510416e725471f8f3100cb34257be74bef3e40ca78460","size_bytes":9569},{"path":"synaptic_tuner/api/v1/providers.py","sha256":"eca5efd5a6dee453543ae338ffddf9451d34ce1743e4ab4bde024b47be571ec0","size_bytes":4940},{"path":"synaptic_tuner/api/v1/results.py","sha256":"65e530325c92c7afa18e3ca5f2ad9a11baf7d590221f08c33badfbf2d5f2c7b2","size_bytes":2236},{"path":"synaptic_tuner/api/v1/runs_facade.py","sha256":"bf6155726c99cfc5574385009e20f4f65b69a9ad4b96e33cf7065263d1d4c455","size_bytes":23572},{"path":"synaptic_tuner/api/v1/training_facade.py","sha256":"bf094d0e152f5861bf8a8ff51962e0700123d9b43ed91d914b9e78bbea6facd9","size_bytes":11118},{"path":"synaptic_tuner/api/v1/training_input.py","sha256":"209326325f8796ab7d038878e92def2c5521d7d98b7b9cc4dd1766d2e4f68c9d","size_bytes":38650},{"path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"path":"tuner/cloud/runtime_layout.py","sha256":"2a4cd1f5fc63d95bb3f0b5bc4f44e616bb0308d9509aceb198943953746ff705","size_bytes":9435},{"path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"path":"tuner/execution/_effect_executor.py","sha256":"7aa3c523d971486417fe735c62a5cb3fe7e0698caa514d0da047293ad6b3e9cd","size_bytes":500},{"path":"tuner/execution/broker.py","sha256":"c504bd734b41ef9785fe5fa591966bb0b0bce3b43dcce0695b4ee0c142ca3d1e","size_bytes":6305},{"path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"path":"tuner/execution/coordinator_v1/__init__.py","sha256":"453718a1827cfb23e7ca0c908d3d256f62ca3e3fff8f75ed8a785f8336b17bbe","size_bytes":66},{"path":"tuner/execution/coordinator_v1/coordinator.py","sha256":"9bc8ecb60d472ad0b3c0d3192d8c73d88eb28dc43af55677fbdd4b3673b71ef9","size_bytes":47018},{"path":"tuner/execution/coordinator_v1/cursors.py","sha256":"5f7c899064dd26959aeed6b328b236b01f7ff549e2d29348bd45dc0480ede26b","size_bytes":9088},{"path":"tuner/execution/coordinator_v1/foundation.py","sha256":"4433c390075544df5516a5e213af4b4b697485ff7627f5a6dcace21a98029f94","size_bytes":12972},{"path":"tuner/execution/coordinator_v1/model.py","sha256":"1d2150d588f6b84f1b58913df398365a9491d6486a8a871bb2a22fa98cafc722","size_bytes":75837},{"path":"tuner/execution/coordinator_v1/ports.py","sha256":"7a028c32c3a3eab905bbc202248b4bf39411cd22e73fddacf5099b7d15dc3832","size_bytes":9343},{"path":"tuner/execution/coordinator_v1/state_machine.py","sha256":"e5e63db3a85fb9fc5b1d7bb5444691f88f308cfb0428842c0801b06d6f2fbb3a","size_bytes":50176},{"path":"tuner/execution/coordinator_v1/stores.py","sha256":"3a875f9e9d7d7d9899d5deba2f4d68cfada12bea499d5897592c9383ba2b16dc","size_bytes":39045},{"path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"path":"tuner/execution/foundation_v2/__init__.py","sha256":"1651bf897879603a83e7824f60d369d52ac5d0308c96f988c24465f33112cd77","size_bytes":77},{"path":"tuner/execution/foundation_v2/authority.py","sha256":"a0eb5b3537a59c7b4d33e9b8617abd072376545d808f1989f562704dfa1ad024","size_bytes":9958},{"path":"tuner/execution/foundation_v2/canonical.py","sha256":"80620a6088d137bfacb210e000c406632bf569de66350628fb0e0aee612babfb","size_bytes":4047},{"path":"tuner/execution/foundation_v2/commands.py","sha256":"0fff7f05401094a60a390514622bbaa5649a8fb1b5f85f938e856192dac34e5c","size_bytes":8891},{"path":"tuner/execution/foundation_v2/executors.py","sha256":"27a428368c7aa4b9b502ae8028de03b48b817ea9d237a08dd112e7a6afc20454","size_bytes":5331},{"path":"tuner/execution/foundation_v2/identities.py","sha256":"720ae3938627f075c181eb5a125d118df87c9d6f7f24869107c74df9fbc44388","size_bytes":2794},{"path":"tuner/execution/foundation_v2/observations.py","sha256":"3642e4fd35478be5cb9a41df8f758ec4581d9f341f470bad1a4a2fbf8093636c","size_bytes":2566},{"path":"tuner/execution/foundation_v2/operations.py","sha256":"0f3c12d916a71a1fe20017b5ff7e99676c0acbf8bae7fac903d629c1aa3a9e6d","size_bytes":2259},{"path":"tuner/execution/foundation_v2/preparation.py","sha256":"09883f2455e521a7baa9b30fbb81518aa33ab2845392009d87cb54606ff80f1d","size_bytes":3575},{"path":"tuner/execution/foundation_v2/receipts.py","sha256":"c327476257236209ea1e29258fa55d90859bb262d27c132870d21399dfb5f626","size_bytes":15339},{"path":"tuner/execution/foundation_v2/references.py","sha256":"53d8e2070395a4c7f5c5a9361532f7a78de714003c760b55a23366394561b37f","size_bytes":3336},{"path":"tuner/execution/foundation_v2/repository.py","sha256":"d311b21fe8f2ed4b85d50ffa55fca370fe6452841d7a0189c5d56b841f69ef7f","size_bytes":57985},{"path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"path":"tuner/execution/operation.py","sha256":"9f86c585fc7ac3f7b2689a4701d598e32a382c3d1e0b87a7732ef35235568504","size_bytes":13702},{"path":"tuner/execution/providers/__init__.py","sha256":"8848731f5d1877c77ea8fcb66a82b85196aeafc06ba0e7463eca291ec1a5f41d","size_bytes":252},{"path":"tuner/execution/providers/contracts.py","sha256":"a21f22c8d373037d6426cd94dda48d7252438a08e115120188980dcaa34d6361","size_bytes":11506},{"path":"tuner/execution/providers/modal/__init__.py","sha256":"0f4e16bed9b4d05c59a5eef1d0ddc061720d5817a68a2bbe1073b20a19c27319","size_bytes":104},{"path":"tuner/execution/providers/modal/binding.py","sha256":"2d2acb7685798f487988b7690c1dbaf6ddc49e34ed8cd3600add438e17fc9ad5","size_bytes":1810},{"path":"tuner/execution/providers/modal/config.py","sha256":"bfa32ed26f3b8ece5e89e1a3524cb81e85ae3240e72ce93430ffb82dbb02f525","size_bytes":20788},{"path":"tuner/execution/providers/modal/contracts.py","sha256":"249e75a9ad13b93c65ac95cb100dc4fe24ed4c59674770832af3d5a9c4112dd2","size_bytes":6449},{"path":"tuner/execution/providers/modal/control.py","sha256":"19e05126196be5009c2cedac36a39008e82ca0bd17c9ec461f68f6ef26ece750","size_bytes":10359},{"path":"tuner/execution/providers/modal/coordinator_adapter.py","sha256":"619ba5e4e10f1a26ad4c154490da9328f9863581df02bf768e48e045ba46e413","size_bytes":13178},{"path":"tuner/execution/providers/modal/coordinator_binding.py","sha256":"e9267c37dce7273ffcad5f1db5332e205ba5282a9fbad4a647581e23c08b821e","size_bytes":4125},{"path":"tuner/execution/providers/modal/coordinator_bundle.py","sha256":"5d8a10d2c4a0dce83174eeca202c377730820ced6945be6f7717e3a0cfd91eb0","size_bytes":31316},{"path":"tuner/execution/providers/modal/coordinator_dispatch.py","sha256":"8e5bc475c6b7d15345e22e1f8cebdabcffab5a01ffefb7d4acbfb073be02b49f","size_bytes":10571},{"path":"tuner/execution/providers/modal/coordinator_launch.py","sha256":"5975bee2d73dd09e02008ede368ebc8f03325815e8bcfd4c093736214d5f0956","size_bytes":10982},{"path":"tuner/execution/providers/modal/coordinator_preflight.py","sha256":"f452e906d37fa9dba932e9df46f829447c95ec276191e34ec3e46831c89da875","size_bytes":13746},{"path":"tuner/execution/providers/modal/coordinator_reader.py","sha256":"92a1c21725587a88435d30b6b687f63711a12ad4c77345f0776521d6503103f4","size_bytes":28524},{"path":"tuner/execution/providers/modal/coordinator_staging.py","sha256":"de47b688e859ab96a612c01b1c7869d814e40b83ae81dcd7f2b1132e18c794f0","size_bytes":20880},{"path":"tuner/execution/providers/modal/coordinator_submit_preparation.py","sha256":"28814779c7f086016ae3ff617e0f1d4cbd8b52f518d3d889a905ea03d70456ec","size_bytes":5213},{"path":"tuner/execution/providers/modal/coordinator_wire.py","sha256":"07d46b92b33e30cb45bb6b1139ce4f80afdb09e1a17a3f3c2526ddcfae6983c9","size_bytes":12998},{"path":"tuner/execution/providers/modal/deployment_identity.py","sha256":"1b7007e5a9c11abdfe0ad4f106f742faa0ee556921b4017d3b48cb772bd0da4f","size_bytes":1196},{"path":"tuner/execution/providers/modal/deployment_v1.py","sha256":"579ace8adbddfe30e09d5a39df9926a54b111a139c704fc7efc7cebf533acf94","size_bytes":3628},{"path":"tuner/execution/providers/modal/facade.py","sha256":"55b89b3d0faff6238eeead068d6963d26f27749d725d2f2b8ae32449b348e32d","size_bytes":17704},{"path":"tuner/execution/providers/modal/inference_artifacts.py","sha256":"5816f0c5662de8edad752554ba149d9527ced40ebdceeccc993c136ab099bff3","size_bytes":14973},{"path":"tuner/execution/providers/modal/inference_binding.py","sha256":"f2e8c4f1d8fc1f5e70702c88cbfff044a1f4f569e06b9a31c33a0cb9aaaf5894","size_bytes":23363},{"path":"tuner/execution/providers/modal/inference_bootstrap.py","sha256":"db2f09c1dfb2aee667b46fbb6566be8b2922d8e762896e5577d5c5078ad7d122","size_bytes":12546},{"path":"tuner/execution/providers/modal/inference_channel.py","sha256":"23492b4c7f48d86d23fb21ffd3902ac3f48bae381c30ea4759c803577911592e","size_bytes":13438},{"path":"tuner/execution/providers/modal/inference_commands.py","sha256":"1fa2dd667cb3b33e012d37b0688bfe08fc7a7cec52230a202f1f9ddda70110c5","size_bytes":4977},{"path":"tuner/execution/providers/modal/inference_entrypoint.py","sha256":"6ec8bdcbbf6d87fb5fc02a2c42d2c0f344ea965ccc4c0d94a630055c6e779991","size_bytes":10873},{"path":"tuner/execution/providers/modal/inference_model.py","sha256":"c08e1a35bb71a11c3466aedfbd8d3de87cf32d6cbd85820c3dcde2fefd14fd65","size_bytes":5935},{"path":"tuner/execution/providers/modal/inference_preparation.py","sha256":"43a1c883a8e5b2f25364e3a20f59d92a852af15fda0c9d506c213c0bf185fc32","size_bytes":49436},{"path":"tuner/execution/providers/modal/inference_runtime.py","sha256":"f8ba0d5744f90ca95e1fea3e5faf293e153a91047ffb09596f0305e1339d56ee","size_bytes":19283},{"path":"tuner/execution/providers/modal/inference_wire.py","sha256":"0492c10c6a384596bba04b3b409dc34f1f1a4afa88b7321d0d1a6fac5e29a1fb","size_bytes":20150},{"path":"tuner/execution/providers/modal/inference_worker.py","sha256":"cfb27e938c60757ef08c80b918fb522012c84918e7a0cd41a63de2cb2486cb87","size_bytes":15657},{"path":"tuner/execution/providers/modal/inference_workload.py","sha256":"4eaf399485e76829a98f57907c012658ce41d02745978827f21262ce2be94ff8","size_bytes":20421},{"path":"tuner/execution/providers/modal/manifest.py","sha256":"fc2ab4dcc46d136a681bf124d7a9fc3e5deedeeb3641d798fd207cdb62bb506f","size_bytes":8579},{"path":"tuner/execution/providers/modal/model_snapshot.py","sha256":"9d4762b821b2944a8a62932d0aec105c2f6920d64bf4ceed6907e5458799e337","size_bytes":14166},{"path":"tuner/execution/providers/modal/mounted_io.py","sha256":"e49da2e749d3be1ca19fdc7f581b29fb6d1e748a86fffcf0fde2be5cdc1a0d5b","size_bytes":13076},{"path":"tuner/execution/providers/modal/prepared_input.py","sha256":"c16cef5cfcc73ab9a7826977b26e18912ac46409d91b9eae165893ed265896ac","size_bytes":8597},{"path":"tuner/execution/providers/modal/resolution.py","sha256":"5c7268aa644103ae0712b03ed55918527d6e2ed6f4120ef169866b0ffd1cbf6f","size_bytes":30179},{"path":"tuner/execution/providers/modal/runtime.py","sha256":"ada13733021a6dab19331da80076df630055abc5bddc192a34a5cda65ac718c4","size_bytes":14810},{"path":"tuner/execution/providers/modal/worker_ports.py","sha256":"adf334368dbb75e40177078cd32aedc445cd9dde8b5ece2401319d472dbccdb0","size_bytes":2928},{"path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"path":"tuner/inference/__init__.py","sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","size_bytes":0},{"path":"tuner/inference/chat_session.py","sha256":"b95a829f834406a0a0c152f9af713225fe205a9225ad7b0f959b1ba801dbcac2","size_bytes":22355},{"path":"tuner/inference/owned_process.py","sha256":"f4cbb4e1c2305615b2c127e1103f60b1e17e955fe20a2a847b1d88b7ba8c4c98","size_bytes":8832},{"path":"tuner/inference/retrieved_model.py","sha256":"4a8cf2b2cbe4f03bf7c4d240c5e59a6ec6d20826c686b3a06c0bc1642fa234d8","size_bytes":31703},{"path":"tuner/inference/run_chat.py","sha256":"9263d325224bee4b9a0e7c16045e30766d9a2e2d2b550f0508040161687d3cfc","size_bytes":6228},{"path":"tuner/inference/serving_target.py","sha256":"50ec8054d1d6d08724d7a29183ca54a04d9c915d2c04274bdac6834764bdb800","size_bytes":12227},{"path":"tuner/inference/vllm_runtime.py","sha256":"fafbcfdeee78143a2760e5fbf29621897e9be014814eefb69f1167b188c296c1","size_bytes":26019},{"path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"path":"tuner/runtime/dispatch.py","sha256":"4c41faf87d7e620bdc281490bf3725550b83dacca8b15ea47bab253f669cdd8e","size_bytes":35463},{"path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"path":"tuner/runtime/verification.py","sha256":"4c2314b20cbf7937a3f24aa90676000b9e21f0ab6701b2c1225a53e5596bf4a0","size_bytes":53856},{"path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"path":"tuner/training/contracts.py","sha256":"91c4d66d9810cc0471981c3d4ef4b54feaed5b92eb41a468f31201f8111303b5","size_bytes":21504},{"path":"tuner/training/coordinator_material.py","sha256":"c563aa3b1fbd940ddd3b8e56e4f33de70675cb831e938175f40debc0645fd9c7","size_bytes":12950},{"path":"tuner/training/grpo_compilation.py","sha256":"c6268486375bbc16f1888bb43d07b93a8de280d000a774914d59d7fd05ca5cc0","size_bytes":8998},{"path":"tuner/training/methods/__init__.py","sha256":"99db689246273707ae3df5087859ebb510e84b664f9f173c0d5827abc3e6a3ea","size_bytes":361},{"path":"tuner/training/methods/sft.py","sha256":"b501f9c214dec4b0b1550543281c25aba8dbb102604df253c641c85f7cae587b","size_bytes":9997},{"path":"tuner/training/packaged_compilation.py","sha256":"477caede6902fc587af680889b93fd9e0e42493d1a6653f3cd7413a9c7cadde8","size_bytes":9413},{"path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722},{"path":"tuner/training/post_training.py","sha256":"ba8c2ea3fe49a647d502c6a874a44658cc2e575d4cd5729966102b0ebb973a88","size_bytes":7971},{"path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"payload_bytes":1656803,"schema_version":"synaptic-modal-inference-worker-closure/v1"} +{"closure_digest":"b05f466d1688ef62eaf11e392bae56ba92ed724c398f26039b1b8aab453f77bb","entrypoint":"tuner/execution/providers/modal/inference_bootstrap.py","member_count":124,"members":[{"path":"Evaluator/__init__.py","sha256":"d1b5aea793a95c47618d4b2469a8bae0b14fd11f81501c1d71546469a39a5cbd","size_bytes":3061},{"path":"Evaluator/base_client.py","sha256":"ac7e77909bdf161a4fd9b1ad2e1e2cfe085ec31f36baa6b19f8976f7b2358cbd","size_bytes":19362},{"path":"Evaluator/config.py","sha256":"94fbe8e198d065141ff9ad002b0a6e26e56d865fc1e3a3fb23479a8e69d09303","size_bytes":21956},{"path":"Evaluator/enums.py","sha256":"2ced9426d860e9ec410e376d9cf853d75e8e111e78dd0bee31e25bee295e4854","size_bytes":3371},{"path":"Evaluator/openai_compat_client.py","sha256":"6ac6211185d2c57d3b31022d31478e3253046fd6cb4de1903d8d0b209bad3a79","size_bytes":5174},{"path":"Evaluator/protocols.py","sha256":"239fe049b5fe4c9e9861aede487feac1f06a7642891378a1ccfba444072ad706","size_bytes":4560},{"path":"Evaluator/verified_vllm_chat.py","sha256":"f130e47d661aa77d68c104084fafe9e5df3af88684ef5a3139f4d53ff2e32465","size_bytes":5916},{"path":"Evaluator/vllm_client.py","sha256":"4e88d2055dc1b32cc22a1393cdf038ccd76d91d915a8be0774b0a8ec87bd4d1e","size_bytes":1995},{"path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"path":"synaptic_tuner/api/v1/planning.py","sha256":"e715b105a513fa56cfa510416e725471f8f3100cb34257be74bef3e40ca78460","size_bytes":9569},{"path":"synaptic_tuner/api/v1/providers.py","sha256":"eca5efd5a6dee453543ae338ffddf9451d34ce1743e4ab4bde024b47be571ec0","size_bytes":4940},{"path":"synaptic_tuner/api/v1/results.py","sha256":"65e530325c92c7afa18e3ca5f2ad9a11baf7d590221f08c33badfbf2d5f2c7b2","size_bytes":2236},{"path":"synaptic_tuner/api/v1/runs_facade.py","sha256":"bf6155726c99cfc5574385009e20f4f65b69a9ad4b96e33cf7065263d1d4c455","size_bytes":23572},{"path":"synaptic_tuner/api/v1/training_facade.py","sha256":"bf094d0e152f5861bf8a8ff51962e0700123d9b43ed91d914b9e78bbea6facd9","size_bytes":11118},{"path":"synaptic_tuner/api/v1/training_input.py","sha256":"209326325f8796ab7d038878e92def2c5521d7d98b7b9cc4dd1766d2e4f68c9d","size_bytes":38650},{"path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"path":"tuner/cloud/runtime_layout.py","sha256":"2a4cd1f5fc63d95bb3f0b5bc4f44e616bb0308d9509aceb198943953746ff705","size_bytes":9435},{"path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"path":"tuner/execution/_effect_executor.py","sha256":"7aa3c523d971486417fe735c62a5cb3fe7e0698caa514d0da047293ad6b3e9cd","size_bytes":500},{"path":"tuner/execution/broker.py","sha256":"c504bd734b41ef9785fe5fa591966bb0b0bce3b43dcce0695b4ee0c142ca3d1e","size_bytes":6305},{"path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"path":"tuner/execution/coordinator_v1/__init__.py","sha256":"453718a1827cfb23e7ca0c908d3d256f62ca3e3fff8f75ed8a785f8336b17bbe","size_bytes":66},{"path":"tuner/execution/coordinator_v1/coordinator.py","sha256":"9bc8ecb60d472ad0b3c0d3192d8c73d88eb28dc43af55677fbdd4b3673b71ef9","size_bytes":47018},{"path":"tuner/execution/coordinator_v1/cursors.py","sha256":"5f7c899064dd26959aeed6b328b236b01f7ff549e2d29348bd45dc0480ede26b","size_bytes":9088},{"path":"tuner/execution/coordinator_v1/foundation.py","sha256":"4433c390075544df5516a5e213af4b4b697485ff7627f5a6dcace21a98029f94","size_bytes":12972},{"path":"tuner/execution/coordinator_v1/model.py","sha256":"1d2150d588f6b84f1b58913df398365a9491d6486a8a871bb2a22fa98cafc722","size_bytes":75837},{"path":"tuner/execution/coordinator_v1/ports.py","sha256":"7a028c32c3a3eab905bbc202248b4bf39411cd22e73fddacf5099b7d15dc3832","size_bytes":9343},{"path":"tuner/execution/coordinator_v1/state_machine.py","sha256":"e5e63db3a85fb9fc5b1d7bb5444691f88f308cfb0428842c0801b06d6f2fbb3a","size_bytes":50176},{"path":"tuner/execution/coordinator_v1/stores.py","sha256":"3a875f9e9d7d7d9899d5deba2f4d68cfada12bea499d5897592c9383ba2b16dc","size_bytes":39045},{"path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"path":"tuner/execution/foundation_v2/__init__.py","sha256":"1651bf897879603a83e7824f60d369d52ac5d0308c96f988c24465f33112cd77","size_bytes":77},{"path":"tuner/execution/foundation_v2/authority.py","sha256":"a0eb5b3537a59c7b4d33e9b8617abd072376545d808f1989f562704dfa1ad024","size_bytes":9958},{"path":"tuner/execution/foundation_v2/canonical.py","sha256":"80620a6088d137bfacb210e000c406632bf569de66350628fb0e0aee612babfb","size_bytes":4047},{"path":"tuner/execution/foundation_v2/commands.py","sha256":"0fff7f05401094a60a390514622bbaa5649a8fb1b5f85f938e856192dac34e5c","size_bytes":8891},{"path":"tuner/execution/foundation_v2/executors.py","sha256":"27a428368c7aa4b9b502ae8028de03b48b817ea9d237a08dd112e7a6afc20454","size_bytes":5331},{"path":"tuner/execution/foundation_v2/identities.py","sha256":"720ae3938627f075c181eb5a125d118df87c9d6f7f24869107c74df9fbc44388","size_bytes":2794},{"path":"tuner/execution/foundation_v2/observations.py","sha256":"3642e4fd35478be5cb9a41df8f758ec4581d9f341f470bad1a4a2fbf8093636c","size_bytes":2566},{"path":"tuner/execution/foundation_v2/operations.py","sha256":"0f3c12d916a71a1fe20017b5ff7e99676c0acbf8bae7fac903d629c1aa3a9e6d","size_bytes":2259},{"path":"tuner/execution/foundation_v2/preparation.py","sha256":"09883f2455e521a7baa9b30fbb81518aa33ab2845392009d87cb54606ff80f1d","size_bytes":3575},{"path":"tuner/execution/foundation_v2/receipts.py","sha256":"c327476257236209ea1e29258fa55d90859bb262d27c132870d21399dfb5f626","size_bytes":15339},{"path":"tuner/execution/foundation_v2/references.py","sha256":"53d8e2070395a4c7f5c5a9361532f7a78de714003c760b55a23366394561b37f","size_bytes":3336},{"path":"tuner/execution/foundation_v2/repository.py","sha256":"d311b21fe8f2ed4b85d50ffa55fca370fe6452841d7a0189c5d56b841f69ef7f","size_bytes":57985},{"path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"path":"tuner/execution/operation.py","sha256":"9f86c585fc7ac3f7b2689a4701d598e32a382c3d1e0b87a7732ef35235568504","size_bytes":13702},{"path":"tuner/execution/providers/__init__.py","sha256":"8848731f5d1877c77ea8fcb66a82b85196aeafc06ba0e7463eca291ec1a5f41d","size_bytes":252},{"path":"tuner/execution/providers/contracts.py","sha256":"a21f22c8d373037d6426cd94dda48d7252438a08e115120188980dcaa34d6361","size_bytes":11506},{"path":"tuner/execution/providers/modal/__init__.py","sha256":"0f4e16bed9b4d05c59a5eef1d0ddc061720d5817a68a2bbe1073b20a19c27319","size_bytes":104},{"path":"tuner/execution/providers/modal/binding.py","sha256":"2d2acb7685798f487988b7690c1dbaf6ddc49e34ed8cd3600add438e17fc9ad5","size_bytes":1810},{"path":"tuner/execution/providers/modal/config.py","sha256":"bfa32ed26f3b8ece5e89e1a3524cb81e85ae3240e72ce93430ffb82dbb02f525","size_bytes":20788},{"path":"tuner/execution/providers/modal/contracts.py","sha256":"249e75a9ad13b93c65ac95cb100dc4fe24ed4c59674770832af3d5a9c4112dd2","size_bytes":6449},{"path":"tuner/execution/providers/modal/control.py","sha256":"19e05126196be5009c2cedac36a39008e82ca0bd17c9ec461f68f6ef26ece750","size_bytes":10359},{"path":"tuner/execution/providers/modal/coordinator_adapter.py","sha256":"619ba5e4e10f1a26ad4c154490da9328f9863581df02bf768e48e045ba46e413","size_bytes":13178},{"path":"tuner/execution/providers/modal/coordinator_binding.py","sha256":"e9267c37dce7273ffcad5f1db5332e205ba5282a9fbad4a647581e23c08b821e","size_bytes":4125},{"path":"tuner/execution/providers/modal/coordinator_bundle.py","sha256":"5d8a10d2c4a0dce83174eeca202c377730820ced6945be6f7717e3a0cfd91eb0","size_bytes":31316},{"path":"tuner/execution/providers/modal/coordinator_dispatch.py","sha256":"8e5bc475c6b7d15345e22e1f8cebdabcffab5a01ffefb7d4acbfb073be02b49f","size_bytes":10571},{"path":"tuner/execution/providers/modal/coordinator_launch.py","sha256":"5975bee2d73dd09e02008ede368ebc8f03325815e8bcfd4c093736214d5f0956","size_bytes":10982},{"path":"tuner/execution/providers/modal/coordinator_preflight.py","sha256":"f452e906d37fa9dba932e9df46f829447c95ec276191e34ec3e46831c89da875","size_bytes":13746},{"path":"tuner/execution/providers/modal/coordinator_reader.py","sha256":"92a1c21725587a88435d30b6b687f63711a12ad4c77345f0776521d6503103f4","size_bytes":28524},{"path":"tuner/execution/providers/modal/coordinator_staging.py","sha256":"de47b688e859ab96a612c01b1c7869d814e40b83ae81dcd7f2b1132e18c794f0","size_bytes":20880},{"path":"tuner/execution/providers/modal/coordinator_submit_preparation.py","sha256":"28814779c7f086016ae3ff617e0f1d4cbd8b52f518d3d889a905ea03d70456ec","size_bytes":5213},{"path":"tuner/execution/providers/modal/coordinator_wire.py","sha256":"07d46b92b33e30cb45bb6b1139ce4f80afdb09e1a17a3f3c2526ddcfae6983c9","size_bytes":12998},{"path":"tuner/execution/providers/modal/deployment_identity.py","sha256":"1b7007e5a9c11abdfe0ad4f106f742faa0ee556921b4017d3b48cb772bd0da4f","size_bytes":1196},{"path":"tuner/execution/providers/modal/deployment_v1.py","sha256":"579ace8adbddfe30e09d5a39df9926a54b111a139c704fc7efc7cebf533acf94","size_bytes":3628},{"path":"tuner/execution/providers/modal/facade.py","sha256":"55b89b3d0faff6238eeead068d6963d26f27749d725d2f2b8ae32449b348e32d","size_bytes":17704},{"path":"tuner/execution/providers/modal/inference_artifacts.py","sha256":"5816f0c5662de8edad752554ba149d9527ced40ebdceeccc993c136ab099bff3","size_bytes":14973},{"path":"tuner/execution/providers/modal/inference_binding.py","sha256":"f2e8c4f1d8fc1f5e70702c88cbfff044a1f4f569e06b9a31c33a0cb9aaaf5894","size_bytes":23363},{"path":"tuner/execution/providers/modal/inference_bootstrap.py","sha256":"db2f09c1dfb2aee667b46fbb6566be8b2922d8e762896e5577d5c5078ad7d122","size_bytes":12546},{"path":"tuner/execution/providers/modal/inference_channel.py","sha256":"23492b4c7f48d86d23fb21ffd3902ac3f48bae381c30ea4759c803577911592e","size_bytes":13438},{"path":"tuner/execution/providers/modal/inference_commands.py","sha256":"1fa2dd667cb3b33e012d37b0688bfe08fc7a7cec52230a202f1f9ddda70110c5","size_bytes":4977},{"path":"tuner/execution/providers/modal/inference_entrypoint.py","sha256":"6ec8bdcbbf6d87fb5fc02a2c42d2c0f344ea965ccc4c0d94a630055c6e779991","size_bytes":10873},{"path":"tuner/execution/providers/modal/inference_model.py","sha256":"c08e1a35bb71a11c3466aedfbd8d3de87cf32d6cbd85820c3dcde2fefd14fd65","size_bytes":5935},{"path":"tuner/execution/providers/modal/inference_preparation.py","sha256":"43a1c883a8e5b2f25364e3a20f59d92a852af15fda0c9d506c213c0bf185fc32","size_bytes":49436},{"path":"tuner/execution/providers/modal/inference_runtime.py","sha256":"f8ba0d5744f90ca95e1fea3e5faf293e153a91047ffb09596f0305e1339d56ee","size_bytes":19283},{"path":"tuner/execution/providers/modal/inference_wire.py","sha256":"0492c10c6a384596bba04b3b409dc34f1f1a4afa88b7321d0d1a6fac5e29a1fb","size_bytes":20150},{"path":"tuner/execution/providers/modal/inference_worker.py","sha256":"cfb27e938c60757ef08c80b918fb522012c84918e7a0cd41a63de2cb2486cb87","size_bytes":15657},{"path":"tuner/execution/providers/modal/inference_workload.py","sha256":"4eaf399485e76829a98f57907c012658ce41d02745978827f21262ce2be94ff8","size_bytes":20421},{"path":"tuner/execution/providers/modal/manifest.py","sha256":"fc2ab4dcc46d136a681bf124d7a9fc3e5deedeeb3641d798fd207cdb62bb506f","size_bytes":8579},{"path":"tuner/execution/providers/modal/model_snapshot.py","sha256":"9d4762b821b2944a8a62932d0aec105c2f6920d64bf4ceed6907e5458799e337","size_bytes":14166},{"path":"tuner/execution/providers/modal/mounted_io.py","sha256":"e49da2e749d3be1ca19fdc7f581b29fb6d1e748a86fffcf0fde2be5cdc1a0d5b","size_bytes":13076},{"path":"tuner/execution/providers/modal/prepared_input.py","sha256":"c16cef5cfcc73ab9a7826977b26e18912ac46409d91b9eae165893ed265896ac","size_bytes":8597},{"path":"tuner/execution/providers/modal/resolution.py","sha256":"5c7268aa644103ae0712b03ed55918527d6e2ed6f4120ef169866b0ffd1cbf6f","size_bytes":30179},{"path":"tuner/execution/providers/modal/runtime.py","sha256":"ada13733021a6dab19331da80076df630055abc5bddc192a34a5cda65ac718c4","size_bytes":14810},{"path":"tuner/execution/providers/modal/worker_ports.py","sha256":"adf334368dbb75e40177078cd32aedc445cd9dde8b5ece2401319d472dbccdb0","size_bytes":2928},{"path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"path":"tuner/inference/__init__.py","sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","size_bytes":0},{"path":"tuner/inference/chat_session.py","sha256":"b95a829f834406a0a0c152f9af713225fe205a9225ad7b0f959b1ba801dbcac2","size_bytes":22355},{"path":"tuner/inference/owned_process.py","sha256":"f4cbb4e1c2305615b2c127e1103f60b1e17e955fe20a2a847b1d88b7ba8c4c98","size_bytes":8832},{"path":"tuner/inference/retrieved_model.py","sha256":"4a8cf2b2cbe4f03bf7c4d240c5e59a6ec6d20826c686b3a06c0bc1642fa234d8","size_bytes":31703},{"path":"tuner/inference/run_chat.py","sha256":"9263d325224bee4b9a0e7c16045e30766d9a2e2d2b550f0508040161687d3cfc","size_bytes":6228},{"path":"tuner/inference/serving_target.py","sha256":"50ec8054d1d6d08724d7a29183ca54a04d9c915d2c04274bdac6834764bdb800","size_bytes":12227},{"path":"tuner/inference/vllm_runtime.py","sha256":"fafbcfdeee78143a2760e5fbf29621897e9be014814eefb69f1167b188c296c1","size_bytes":26019},{"path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"path":"tuner/runtime/dispatch.py","sha256":"4c41faf87d7e620bdc281490bf3725550b83dacca8b15ea47bab253f669cdd8e","size_bytes":35463},{"path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"path":"tuner/runtime/verification.py","sha256":"4c2314b20cbf7937a3f24aa90676000b9e21f0ab6701b2c1225a53e5596bf4a0","size_bytes":53856},{"path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"path":"tuner/training/contracts.py","sha256":"ebe52af8ba1c02309d39317694bc04547a6491d110c60229282517dbdab5a417","size_bytes":21524},{"path":"tuner/training/coordinator_material.py","sha256":"c563aa3b1fbd940ddd3b8e56e4f33de70675cb831e938175f40debc0645fd9c7","size_bytes":12950},{"path":"tuner/training/grpo_compilation.py","sha256":"c6268486375bbc16f1888bb43d07b93a8de280d000a774914d59d7fd05ca5cc0","size_bytes":8998},{"path":"tuner/training/methods/__init__.py","sha256":"99db689246273707ae3df5087859ebb510e84b664f9f173c0d5827abc3e6a3ea","size_bytes":361},{"path":"tuner/training/methods/sft.py","sha256":"b501f9c214dec4b0b1550543281c25aba8dbb102604df253c641c85f7cae587b","size_bytes":9997},{"path":"tuner/training/packaged_compilation.py","sha256":"477caede6902fc587af680889b93fd9e0e42493d1a6653f3cd7413a9c7cadde8","size_bytes":9413},{"path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722},{"path":"tuner/training/post_training.py","sha256":"ba8c2ea3fe49a647d502c6a874a44658cc2e575d4cd5729966102b0ebb973a88","size_bytes":7971},{"path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"payload_bytes":1656823,"schema_version":"synaptic-modal-inference-worker-closure/v1"} diff --git a/tuner/execution/providers/modal/modal-runtime-v1.lock.json b/tuner/execution/providers/modal/modal-runtime-v1.lock.json index e27d0cca..02a0f28f 100644 --- a/tuner/execution/providers/modal/modal-runtime-v1.lock.json +++ b/tuner/execution/providers/modal/modal-runtime-v1.lock.json @@ -318,7 +318,7 @@ }, "bootstrap__tuner__runtime__manifests__offline-sft-worker-v1_json": { "path": "tuner/runtime/manifests/offline-sft-worker-v1.json", - "sha256": "3842689403abc5b42e28d58545b88616b81cd8fc738ee5557e97159df65d00ef" + "sha256": "9de89c12761c392e0cd2babbd85351d96ccd994b07d80d8662f261ab107c03b7" }, "bootstrap__tuner__runtime__offline_sft_worker_py": { "path": "tuner/runtime/offline_sft_worker.py", @@ -330,7 +330,7 @@ }, "bootstrap__tuner__training__contracts_py": { "path": "tuner/training/contracts.py", - "sha256": "91c4d66d9810cc0471981c3d4ef4b54feaed5b92eb41a468f31201f8111303b5" + "sha256": "ebe52af8ba1c02309d39317694bc04547a6491d110c60229282517dbdab5a417" }, "bootstrap__tuner__training__coordinator_material_py": { "path": "tuner/training/coordinator_material.py", diff --git a/tuner/runtime/manifests/offline-grpo-worker-v1.json b/tuner/runtime/manifests/offline-grpo-worker-v1.json index ef21c0e8..50a54b89 100644 --- a/tuner/runtime/manifests/offline-grpo-worker-v1.json +++ b/tuner/runtime/manifests/offline-grpo-worker-v1.json @@ -1 +1 @@ -{"closure_digest":"bfcc29f41d774cb393545417e5a43751776c16aa2efd9bcd39b09371d609ba87","closure_ref":"synaptic/offline-grpo-worker/v1","entrypoint":"tuner/runtime/packaged_grpo_child.py","member_count":131,"members":[{"git_mode":"100644","path":"SynthChat/__init__.py","sha256":"f9941fd4b7c15654b950b3d107ac9a0373d1ddfbe9dadcc72f8f608bfbcfa263","size_bytes":1583},{"git_mode":"100644","path":"SynthChat/config/__init__.py","sha256":"b9b4a831d9e625e63872dbd9d81fedf54064fdb2546c2cbdd3e9dde65c59a1e8","size_bytes":1162},{"git_mode":"100644","path":"SynthChat/config/config_loader.py","sha256":"09f80fd591fb9bf22afd712f613fa41d2f78b8e54f748c07d8e168082eed8fc1","size_bytes":9173},{"git_mode":"100644","path":"SynthChat/config/format_resolver.py","sha256":"264f6eeffe4068b554828e1cbc0c098eac9172dcd5fd0437c0e94a87a6c154c1","size_bytes":11651},{"git_mode":"100644","path":"SynthChat/utils/__init__.py","sha256":"0481a87cd831f9d77f6401fff01225e07ceb6295a99ce9db6391f6dfb86fc985","size_bytes":112},{"git_mode":"100644","path":"SynthChat/utils/prompt_renderer.py","sha256":"1f1479fe5ac9edeb33874422da79166abca9c582a34ebea6a532cccbd5aa15eb","size_bytes":1442},{"git_mode":"100644","path":"SynthChat/utils/yaml_loader.py","sha256":"125977de23485ea558d54744bf7bb8bb933aeae446eaae6be3f0ec09f1c8adbc","size_bytes":5264},{"git_mode":"100644","path":"Trainers/grpo/configs/config.yaml","sha256":"3346a0530959169e9d556e37b0342ba6ed93c2831164b6da225b3054d030b1e7","size_bytes":3546},{"git_mode":"100644","path":"Trainers/grpo/runtime_v1.py","sha256":"d71e2c2d68753d4ebc319ecbffd0676812278d32ad8e827e5361af1dafd94c72","size_bytes":45029},{"git_mode":"100644","path":"Trainers/grpo/src/data_loader.py","sha256":"0b9357ce4239331219b2889715c2bef6d48ef0af73bf886636eda31bca92a9ba","size_bytes":4101},{"git_mode":"100644","path":"Trainers/grpo/src/model_loader.py","sha256":"78cb7b80be05bc3f16fd05866973cd4324eff9454acea89eacb30ad37776d00a","size_bytes":7927},{"git_mode":"100644","path":"Trainers/grpo/src/rewards.py","sha256":"e157997b56cf46f0fd270d75c43c4e6b3f4b7c9f8e1a03e5e90f1cec31c18578","size_bytes":23285},{"git_mode":"100644","path":"Trainers/sft/runtime_v1.py","sha256":"460121aaee3eb83764f67d9307cfb57a3b25c117611d987577b9631b6956a1b3","size_bytes":104495},{"git_mode":"100644","path":"shared/judge/__init__.py","sha256":"08fb40e578eae3a0716dcb66664aafc9b865a02739b8777a15ac9fcd51a41848","size_bytes":870},{"git_mode":"100644","path":"shared/judge/interaction_logger.py","sha256":"41ce3ea4be06b0fdd7f2d40e5ea8f8028022fe18f912d460ae06b4c37bdb2642","size_bytes":5573},{"git_mode":"100644","path":"shared/judge/judge_service.py","sha256":"87950058f236df537d5d2921f0d9e55c4eebdc9ffe46555a3874c23d9cb1e91e","size_bytes":17998},{"git_mode":"100644","path":"shared/judge/models.py","sha256":"0c1d18ab9e69c851596d9e2e23ffe7a73cf994ee106fa70b64b8bc698de3e591","size_bytes":8459},{"git_mode":"100644","path":"shared/judge/rubric_loader.py","sha256":"84584eb9f58e8e60e78dd174412bd7d6baae96a5772b2933fe020e54364065c3","size_bytes":4191},{"git_mode":"100644","path":"shared/judge/schema_builder.py","sha256":"523b0dafc8442d92fbb68b758723067be16a698968028a76e40b470e78f73f16","size_bytes":3062},{"git_mode":"100644","path":"shared/llm/__init__.py","sha256":"44b0b934213c3f1facda937ff5fd0fd97411fd335053ba9c178e230f9b06b4ed","size_bytes":1920},{"git_mode":"100644","path":"shared/llm/base.py","sha256":"bd0cbaf0f4edbc188e511ea8fd4401cc1ec3f937681ff126c992d228009ee61e","size_bytes":9185},{"git_mode":"100644","path":"shared/llm/config.py","sha256":"a3f95c1414898860fd40bd9f2a90fc209bb1b0955d0402cb6da67bbfaf1f1420","size_bytes":9822},{"git_mode":"100644","path":"shared/llm/exceptions.py","sha256":"7223e9c03f111b10c760b4c9d7c22df5958c855a5076de0d74872d1dc6ca83f2","size_bytes":589},{"git_mode":"100644","path":"shared/llm/factory.py","sha256":"11263dcc4803ffecdbbfd343a2dbf574bfd4a4efb51f60355699be77f7871b35","size_bytes":4583},{"git_mode":"100644","path":"shared/llm/providers/__init__.py","sha256":"e0c78fec1fa6ab46068760fe2f4ef9af4d6b2645adc80fe75bbdd06039d19138","size_bytes":310},{"git_mode":"100644","path":"shared/llm/providers/lmstudio.py","sha256":"64f70a0a1ccfd67467eb026d6be495881cd53eee1fc267ee6ea059ee8b1f6282","size_bytes":7871},{"git_mode":"100644","path":"shared/llm/providers/ollama.py","sha256":"9b646ce7631dfdb6243e2660a8257d84ab2711c2fe3cb5b408466ff4f1aa780d","size_bytes":7020},{"git_mode":"100644","path":"shared/llm/providers/openai_responses.py","sha256":"70ca5fa422f1a9a29ed153609c124237c8c6d2907885972948bd00c2a9b8e623","size_bytes":10346},{"git_mode":"100644","path":"shared/llm/providers/openrouter.py","sha256":"42276fbed027f1d98183e17ad7a37e87296fe57ea73125a687cb78738dd07dda","size_bytes":17541},{"git_mode":"100644","path":"shared/llm/providers/unsloth.py","sha256":"4d60fc82e70a8f8296dcd231a68754913a3de55b8148d26de831cc177f95b137","size_bytes":11619},{"git_mode":"100644","path":"shared/llm/usage.py","sha256":"9a9d27e8689304e58d1290445bfbbe27b476b437584c6357d894b53119bd91e6","size_bytes":6964},{"git_mode":"100644","path":"shared/ml/__init__.py","sha256":"ee72d716daaba9ee99ecc8be99d776fd695436c34c5768b3e83f51f407ccf71b","size_bytes":1262},{"git_mode":"100644","path":"shared/ml/retrieval_metrics.py","sha256":"67be37fea3634cca2e7c8f42c87f2e4c207714279cece0b1fe2e9f94088ebda0","size_bytes":11456},{"git_mode":"100644","path":"shared/model_loading/__init__.py","sha256":"19e82f3112b3685a95b2f8260c5f11862d649d23968cfa6eefe306319d95a51e","size_bytes":1755},{"git_mode":"100644","path":"shared/model_loading/merge.py","sha256":"e8f0f013076b276a0f722486cb6d7583c1f9b77cde4f716a4d838ef909406711","size_bytes":9529},{"git_mode":"100644","path":"shared/training_utils.py","sha256":"15136abdb987a7bdfbc8917c49f9c77ff553e91ce53890e239ea629241345a65","size_bytes":28814},{"git_mode":"100644","path":"shared/utilities/__init__.py","sha256":"546349addf2ba6c72661e185477354d743f6a42e8ef3bf8e78ed19a17b85e2ab","size_bytes":297},{"git_mode":"100644","path":"shared/utilities/env.py","sha256":"06f56639bc506d4e497ee79d4393d17d499bfd561fd0d0679c0ea7e103b8e886","size_bytes":3318},{"git_mode":"100644","path":"shared/utilities/paths.py","sha256":"3bfa1a13e4deef02cd91a920410e839f8ab0d34ec1081903cbcb742589e995b4","size_bytes":6843},{"git_mode":"100644","path":"shared/utilities/yaml_loader.py","sha256":"203996ff297e1714b0a3e40446f719e24b6d0e20eddb91bc528100b497d8591c","size_bytes":679},{"git_mode":"100644","path":"shared/validation/__init__.py","sha256":"2c80d0a16e9b08c649b131deec3e32cfe9260d127c3b6e9c4a6508eb730e0e7f","size_bytes":1828},{"git_mode":"100644","path":"shared/validation/fitness.py","sha256":"6d90c803c59afa2e1054f33eac06c28a0d765160b53c090fd8b3bad4568cea48","size_bytes":8227},{"git_mode":"100644","path":"shared/validation/parsing/__init__.py","sha256":"e04b52ae2417aabdf8f48d80e847d60de41ec094824d4f6d088aaa132209b6c2","size_bytes":1768},{"git_mode":"100644","path":"shared/validation/parsing/configured_formats.py","sha256":"2d69eff733771bd614a8c1e1252511041fdd4bd5a13cf565205e1ebfb52dc88c","size_bytes":8162},{"git_mode":"100644","path":"shared/validation/parsing/enums.py","sha256":"4ea46d5ef556cc9d5d93d75f944e5a5120effea5264ccd89318d06228beb82bf","size_bytes":1746},{"git_mode":"100644","path":"shared/validation/parsing/response_parser.py","sha256":"5968f7d4f9f70ca269643276a71c238d6543778a47222724a1db69090b8c3471","size_bytes":21535},{"git_mode":"100644","path":"shared/validation/parsing/tool_call_parser.py","sha256":"e786758dd896bf75ceb49ea46a2a37fbea6d13692d7a1a63a26ae2872e33017a","size_bytes":13943},{"git_mode":"100644","path":"shared/validation/parsing/utilities.py","sha256":"42b1c7cb98807601a46066d5073e4d3e1984f4c848f632e46371b55e25732aa2","size_bytes":3024},{"git_mode":"100644","path":"shared/validation/rollout_filters.py","sha256":"d412c53eacb8e7fe9c459345e3dbf83a65ac50fe413d58d567d41a9c7fa6fc86","size_bytes":18795},{"git_mode":"100644","path":"shared/validation/rubric/__init__.py","sha256":"7df886540f94034002142322a481672a991d3059ec503cb996ec03e42d3123f6","size_bytes":438},{"git_mode":"100644","path":"shared/validation/rubric/rubric_cache.py","sha256":"d428466ebdadafe0c8cd8febab5e1fa49414d846efeb45c70cba47102532c2ad","size_bytes":3121},{"git_mode":"100644","path":"shared/validation/rubric/rubric_loader.py","sha256":"f77ca87d02885899d61db49fe709cba445e5c6d35fccdadb1deb8e052d8141e6","size_bytes":2313},{"git_mode":"100644","path":"shared/validation/rubric/rubric_repository.py","sha256":"1173978ad22f34625e981dc2148525c87c7745a927ee0d81c93bc1a030f56676","size_bytes":7104},{"git_mode":"100644","path":"shared/validation/validators/__init__.py","sha256":"d86436b05056c97a9b0aaae4465d23d03f407e737f4b2098fe3fc274fca64035","size_bytes":1031},{"git_mode":"100644","path":"shared/validation/validators/base.py","sha256":"764ae0d8a1439a4823a7ce82fafd1b69406873549c847b65ed57ecb3da97a264","size_bytes":1564},{"git_mode":"100644","path":"shared/validation/validators/content/__init__.py","sha256":"ed370a9fdc34b511dcfe88053bee6e9d4f30e3f7de2263cd3c675b8021865bb2","size_bytes":600},{"git_mode":"100644","path":"shared/validation/validators/content/code_validator.py","sha256":"19ded618181d16de6fa3667a64cc80a96628fd6d0f39ee5412f22c3e7aa75842","size_bytes":1493},{"git_mode":"100644","path":"shared/validation/validators/content/json_validator.py","sha256":"dd4b6fb4cb51aabaac37c052df9ff23b5051fb575dba2e893458d3d64edba497","size_bytes":7524},{"git_mode":"100644","path":"shared/validation/validators/content/regex_validator.py","sha256":"bea1124ccc79fec327a24a61e96110696c3346529d303ddff4a56ecba34b1513","size_bytes":3148},{"git_mode":"100644","path":"shared/validation/validators/content/xml_validator.py","sha256":"970937f6d8d9ef61594e2a34a2804d999482bf319c237a1891db214250bedd2e","size_bytes":1056},{"git_mode":"100644","path":"shared/validation/validators/content/yaml_validator.py","sha256":"75571df0e7ac750feeb8f92dc7da5bdc4d085e032bcf94650d826ecae07d3219","size_bytes":3738},{"git_mode":"100644","path":"shared/validation/validators/cross_scope_validator.py","sha256":"27ed2f25b96cc6fecd5e0702fdbe98a8248caf1483952a0d2402d96cf56ee269","size_bytes":8323},{"git_mode":"100644","path":"shared/validation/validators/structure_validator.py","sha256":"d637fde147e89d052229f3951ac744b56d87eb3f8553626b61dfd36bbb9829cf","size_bytes":20391},{"git_mode":"100644","path":"shared/verifiers/__init__.py","sha256":"bcc0a64a5b1be388e3dc1e913492506cac4daeb4cb5f7a210bca5bd75bafc352","size_bytes":950},{"git_mode":"100644","path":"shared/verifiers/builtins/__init__.py","sha256":"209700d4779871de25567fb6daeabad23ef3b8791e2f6c2b10749c2beead9c88","size_bytes":777},{"git_mode":"100644","path":"shared/verifiers/builtins/args_match.py","sha256":"19f44e00d8dc78d93e85361c9ecd65790fb6fa31f740f9aab743d36e20f38a0f","size_bytes":14581},{"git_mode":"100644","path":"shared/verifiers/builtins/assertion_verifier.py","sha256":"3ab2e6983adf1c9774e665961466856025fca6905d82094e2f0bc48e67cdfc46","size_bytes":17366},{"git_mode":"100644","path":"shared/verifiers/builtins/audio_verifier.py","sha256":"51c75365d411ab77535e343d771178f16fd6fee133b3d815eab0302c783fd207","size_bytes":16786},{"git_mode":"100644","path":"shared/verifiers/builtins/llm_judge.py","sha256":"b4741581e4388ed43bce5fd4def1465abc9b1eab3b8b4905d7ff07f05dd80d15","size_bytes":6915},{"git_mode":"100644","path":"shared/verifiers/builtins/retrieval_verifier.py","sha256":"b70c804369efa4a76dedeb1aeb829814952fecaecee51352803f100f12ac938d","size_bytes":15290},{"git_mode":"100644","path":"shared/verifiers/builtins/structure.py","sha256":"3030f1ca99e9355b0b37a8772937318ef334857aae5c05e92ad3db1318a4dc26","size_bytes":2167},{"git_mode":"100644","path":"shared/verifiers/builtins/substring.py","sha256":"dd86a419aa78058f5e855a538cf63e8a48c3e757f0988139bae0f231c531d76a","size_bytes":3790},{"git_mode":"100644","path":"shared/verifiers/builtins/tool_sequence.py","sha256":"111cb44e290877f1f265fd3d6d9e3d6635501b06f3648613af4f4f716b9f8d56","size_bytes":6459},{"git_mode":"100644","path":"shared/verifiers/composite.py","sha256":"678dbdb3bbab21f4a4956b7e70e8ef2d5dc883ec500a1d2278103a1f892ef66c","size_bytes":5681},{"git_mode":"100644","path":"shared/verifiers/contract.py","sha256":"c594c5527aa949d02cdf8626d03a5ef958148d647a7d7b84df6ee5c6b6ce56cc","size_bytes":2367},{"git_mode":"100644","path":"shared/verifiers/extraction.py","sha256":"f4a8e734957abb1327eeca7faa7cf5b1eb6d9247180224315e4ea6a42bc2e867","size_bytes":6056},{"git_mode":"100644","path":"shared/verifiers/registry.py","sha256":"f55035cd60ee539e4577ae1e4f101ed4b42d6c4adfb5bc0e17ae415607a3db2b","size_bytes":2475},{"git_mode":"100644","path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"git_mode":"100644","path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"git_mode":"100644","path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"git_mode":"100644","path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"git_mode":"100644","path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"git_mode":"100644","path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"git_mode":"100644","path":"synaptic_tuner/api/v1/observations.py","sha256":"b33f5826e6c03db889e0ff29c3eba65f5eab436b3d3f5146b8e4317828178d07","size_bytes":26569},{"git_mode":"100644","path":"synaptic_tuner/api/v1/planning.py","sha256":"e715b105a513fa56cfa510416e725471f8f3100cb34257be74bef3e40ca78460","size_bytes":9569},{"git_mode":"100644","path":"synaptic_tuner/api/v1/providers.py","sha256":"eca5efd5a6dee453543ae338ffddf9451d34ce1743e4ab4bde024b47be571ec0","size_bytes":4940},{"git_mode":"100644","path":"synaptic_tuner/api/v1/results.py","sha256":"65e530325c92c7afa18e3ca5f2ad9a11baf7d590221f08c33badfbf2d5f2c7b2","size_bytes":2236},{"git_mode":"100644","path":"synaptic_tuner/api/v1/training_facade.py","sha256":"bf094d0e152f5861bf8a8ff51962e0700123d9b43ed91d914b9e78bbea6facd9","size_bytes":11118},{"git_mode":"100644","path":"synaptic_tuner/api/v1/training_input.py","sha256":"209326325f8796ab7d038878e92def2c5521d7d98b7b9cc4dd1766d2e4f68c9d","size_bytes":38650},{"git_mode":"100644","path":"synaptic_tuner/api/v1/training_sources.py","sha256":"4c4ce3bc9d73e03d550c61f5ef8f18099137fe82a9be6ea96b226e7363e984b8","size_bytes":8454},{"git_mode":"100644","path":"synaptic_tuner/api/v1/usage.py","sha256":"33e36965e14ae52037f1da877c6606c46a40edf575e248769d5e5731b47b1307","size_bytes":6086},{"git_mode":"100644","path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"git_mode":"100644","path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"git_mode":"100644","path":"tuner/cloud/runtime_layout.py","sha256":"2a4cd1f5fc63d95bb3f0b5bc4f44e616bb0308d9509aceb198943953746ff705","size_bytes":9435},{"git_mode":"100644","path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"git_mode":"100644","path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"git_mode":"100644","path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"git_mode":"100644","path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"git_mode":"100644","path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"git_mode":"100644","path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"git_mode":"100644","path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"git_mode":"100644","path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"git_mode":"100644","path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"git_mode":"100644","path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"git_mode":"100644","path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"git_mode":"100644","path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"git_mode":"100644","path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"git_mode":"100644","path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"git_mode":"100644","path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"git_mode":"100644","path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"git_mode":"100644","path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"git_mode":"100644","path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"git_mode":"100644","path":"tuner/runtime/dispatch.py","sha256":"4c41faf87d7e620bdc281490bf3725550b83dacca8b15ea47bab253f669cdd8e","size_bytes":35463},{"git_mode":"100644","path":"tuner/runtime/offline_grpo_worker.py","sha256":"aac831048855e959598a4dc45b29e614f06f8714df18e868fd6b069e3c765710","size_bytes":9323},{"git_mode":"100644","path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_child.py","sha256":"124d5b2a969d38ac2e488d58a216759108f1a55dfcc3dcaf1012791efe792e4b","size_bytes":8273},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_execution.py","sha256":"a77c7d6b97ad2ded3b9631174e479e60462f1894513f90fd2fc7798828c6867d","size_bytes":14943},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_verification.py","sha256":"56564c6ee6414a3a71ce7c9ef330152d812fede2a711a2329463a0b81a61b0d6","size_bytes":10319},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_worker_closure.py","sha256":"81da8db5a61e9a874234d1e50c529e1056282fd0eef133b2720eb0f34f78cddc","size_bytes":3735},{"git_mode":"100644","path":"tuner/runtime/packaged_sft_child.py","sha256":"ab41eb7dca28b440c043f589d297dc0d9f718ed14f6cb36c96d7bd1b7bfcdc14","size_bytes":23556},{"git_mode":"100644","path":"tuner/runtime/packaged_sft_execution.py","sha256":"cab2b5032d16146422398305120f52c5a124287ecf929b640a1f17d0ea2ed648","size_bytes":80053},{"git_mode":"100644","path":"tuner/runtime/packaged_training_worker.py","sha256":"9cfe13dc9f0124b4da01d05e92cf3c2c9249d075c3a508d6b93b0a7c7f0e109f","size_bytes":34986},{"git_mode":"100644","path":"tuner/runtime/packaged_worker_closure.py","sha256":"e457937426e19747e867a314c1a2556ee64bd5cbc19359f535253b02403d1443","size_bytes":8705},{"git_mode":"100644","path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"git_mode":"100644","path":"tuner/runtime/verification.py","sha256":"4c2314b20cbf7937a3f24aa90676000b9e21f0ab6701b2c1225a53e5596bf4a0","size_bytes":53856},{"git_mode":"100644","path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"git_mode":"100644","path":"tuner/training/contracts.py","sha256":"91c4d66d9810cc0471981c3d4ef4b54feaed5b92eb41a468f31201f8111303b5","size_bytes":21504},{"git_mode":"100644","path":"tuner/training/grpo_compilation.py","sha256":"c6268486375bbc16f1888bb43d07b93a8de280d000a774914d59d7fd05ca5cc0","size_bytes":8998},{"git_mode":"100644","path":"tuner/training/packaged_compilation.py","sha256":"477caede6902fc587af680889b93fd9e0e42493d1a6653f3cd7413a9c7cadde8","size_bytes":9413},{"git_mode":"100644","path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722},{"git_mode":"100644","path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"optional_features":[],"owned_module_prefixes":["tuner","synaptic_tuner","Trainers","shared","SynthChat","Evaluator","MechInterp","configs","src"],"payload_bytes":1408447,"schema_version":"synaptic-offline-grpo-worker-closure/v1","trainer_entrypoint":"Trainers/grpo/runtime_v1.py"} +{"closure_digest":"37cebe9e2a835dbbfb3da876d0568249d744958a9b4d3957a3d44553940ec8bb","closure_ref":"synaptic/offline-grpo-worker/v1","entrypoint":"tuner/runtime/packaged_grpo_child.py","member_count":131,"members":[{"git_mode":"100644","path":"SynthChat/__init__.py","sha256":"f9941fd4b7c15654b950b3d107ac9a0373d1ddfbe9dadcc72f8f608bfbcfa263","size_bytes":1583},{"git_mode":"100644","path":"SynthChat/config/__init__.py","sha256":"b9b4a831d9e625e63872dbd9d81fedf54064fdb2546c2cbdd3e9dde65c59a1e8","size_bytes":1162},{"git_mode":"100644","path":"SynthChat/config/config_loader.py","sha256":"09f80fd591fb9bf22afd712f613fa41d2f78b8e54f748c07d8e168082eed8fc1","size_bytes":9173},{"git_mode":"100644","path":"SynthChat/config/format_resolver.py","sha256":"264f6eeffe4068b554828e1cbc0c098eac9172dcd5fd0437c0e94a87a6c154c1","size_bytes":11651},{"git_mode":"100644","path":"SynthChat/utils/__init__.py","sha256":"0481a87cd831f9d77f6401fff01225e07ceb6295a99ce9db6391f6dfb86fc985","size_bytes":112},{"git_mode":"100644","path":"SynthChat/utils/prompt_renderer.py","sha256":"1f1479fe5ac9edeb33874422da79166abca9c582a34ebea6a532cccbd5aa15eb","size_bytes":1442},{"git_mode":"100644","path":"SynthChat/utils/yaml_loader.py","sha256":"125977de23485ea558d54744bf7bb8bb933aeae446eaae6be3f0ec09f1c8adbc","size_bytes":5264},{"git_mode":"100644","path":"Trainers/grpo/configs/config.yaml","sha256":"3346a0530959169e9d556e37b0342ba6ed93c2831164b6da225b3054d030b1e7","size_bytes":3546},{"git_mode":"100644","path":"Trainers/grpo/runtime_v1.py","sha256":"d71e2c2d68753d4ebc319ecbffd0676812278d32ad8e827e5361af1dafd94c72","size_bytes":45029},{"git_mode":"100644","path":"Trainers/grpo/src/data_loader.py","sha256":"0b9357ce4239331219b2889715c2bef6d48ef0af73bf886636eda31bca92a9ba","size_bytes":4101},{"git_mode":"100644","path":"Trainers/grpo/src/model_loader.py","sha256":"78cb7b80be05bc3f16fd05866973cd4324eff9454acea89eacb30ad37776d00a","size_bytes":7927},{"git_mode":"100644","path":"Trainers/grpo/src/rewards.py","sha256":"e157997b56cf46f0fd270d75c43c4e6b3f4b7c9f8e1a03e5e90f1cec31c18578","size_bytes":23285},{"git_mode":"100644","path":"Trainers/sft/runtime_v1.py","sha256":"460121aaee3eb83764f67d9307cfb57a3b25c117611d987577b9631b6956a1b3","size_bytes":104495},{"git_mode":"100644","path":"shared/judge/__init__.py","sha256":"08fb40e578eae3a0716dcb66664aafc9b865a02739b8777a15ac9fcd51a41848","size_bytes":870},{"git_mode":"100644","path":"shared/judge/interaction_logger.py","sha256":"41ce3ea4be06b0fdd7f2d40e5ea8f8028022fe18f912d460ae06b4c37bdb2642","size_bytes":5573},{"git_mode":"100644","path":"shared/judge/judge_service.py","sha256":"87950058f236df537d5d2921f0d9e55c4eebdc9ffe46555a3874c23d9cb1e91e","size_bytes":17998},{"git_mode":"100644","path":"shared/judge/models.py","sha256":"0c1d18ab9e69c851596d9e2e23ffe7a73cf994ee106fa70b64b8bc698de3e591","size_bytes":8459},{"git_mode":"100644","path":"shared/judge/rubric_loader.py","sha256":"84584eb9f58e8e60e78dd174412bd7d6baae96a5772b2933fe020e54364065c3","size_bytes":4191},{"git_mode":"100644","path":"shared/judge/schema_builder.py","sha256":"523b0dafc8442d92fbb68b758723067be16a698968028a76e40b470e78f73f16","size_bytes":3062},{"git_mode":"100644","path":"shared/llm/__init__.py","sha256":"44b0b934213c3f1facda937ff5fd0fd97411fd335053ba9c178e230f9b06b4ed","size_bytes":1920},{"git_mode":"100644","path":"shared/llm/base.py","sha256":"bd0cbaf0f4edbc188e511ea8fd4401cc1ec3f937681ff126c992d228009ee61e","size_bytes":9185},{"git_mode":"100644","path":"shared/llm/config.py","sha256":"a3f95c1414898860fd40bd9f2a90fc209bb1b0955d0402cb6da67bbfaf1f1420","size_bytes":9822},{"git_mode":"100644","path":"shared/llm/exceptions.py","sha256":"7223e9c03f111b10c760b4c9d7c22df5958c855a5076de0d74872d1dc6ca83f2","size_bytes":589},{"git_mode":"100644","path":"shared/llm/factory.py","sha256":"11263dcc4803ffecdbbfd343a2dbf574bfd4a4efb51f60355699be77f7871b35","size_bytes":4583},{"git_mode":"100644","path":"shared/llm/providers/__init__.py","sha256":"e0c78fec1fa6ab46068760fe2f4ef9af4d6b2645adc80fe75bbdd06039d19138","size_bytes":310},{"git_mode":"100644","path":"shared/llm/providers/lmstudio.py","sha256":"64f70a0a1ccfd67467eb026d6be495881cd53eee1fc267ee6ea059ee8b1f6282","size_bytes":7871},{"git_mode":"100644","path":"shared/llm/providers/ollama.py","sha256":"9b646ce7631dfdb6243e2660a8257d84ab2711c2fe3cb5b408466ff4f1aa780d","size_bytes":7020},{"git_mode":"100644","path":"shared/llm/providers/openai_responses.py","sha256":"70ca5fa422f1a9a29ed153609c124237c8c6d2907885972948bd00c2a9b8e623","size_bytes":10346},{"git_mode":"100644","path":"shared/llm/providers/openrouter.py","sha256":"42276fbed027f1d98183e17ad7a37e87296fe57ea73125a687cb78738dd07dda","size_bytes":17541},{"git_mode":"100644","path":"shared/llm/providers/unsloth.py","sha256":"4d60fc82e70a8f8296dcd231a68754913a3de55b8148d26de831cc177f95b137","size_bytes":11619},{"git_mode":"100644","path":"shared/llm/usage.py","sha256":"9a9d27e8689304e58d1290445bfbbe27b476b437584c6357d894b53119bd91e6","size_bytes":6964},{"git_mode":"100644","path":"shared/ml/__init__.py","sha256":"ee72d716daaba9ee99ecc8be99d776fd695436c34c5768b3e83f51f407ccf71b","size_bytes":1262},{"git_mode":"100644","path":"shared/ml/retrieval_metrics.py","sha256":"67be37fea3634cca2e7c8f42c87f2e4c207714279cece0b1fe2e9f94088ebda0","size_bytes":11456},{"git_mode":"100644","path":"shared/model_loading/__init__.py","sha256":"19e82f3112b3685a95b2f8260c5f11862d649d23968cfa6eefe306319d95a51e","size_bytes":1755},{"git_mode":"100644","path":"shared/model_loading/merge.py","sha256":"e8f0f013076b276a0f722486cb6d7583c1f9b77cde4f716a4d838ef909406711","size_bytes":9529},{"git_mode":"100644","path":"shared/training_utils.py","sha256":"15136abdb987a7bdfbc8917c49f9c77ff553e91ce53890e239ea629241345a65","size_bytes":28814},{"git_mode":"100644","path":"shared/utilities/__init__.py","sha256":"546349addf2ba6c72661e185477354d743f6a42e8ef3bf8e78ed19a17b85e2ab","size_bytes":297},{"git_mode":"100644","path":"shared/utilities/env.py","sha256":"06f56639bc506d4e497ee79d4393d17d499bfd561fd0d0679c0ea7e103b8e886","size_bytes":3318},{"git_mode":"100644","path":"shared/utilities/paths.py","sha256":"3bfa1a13e4deef02cd91a920410e839f8ab0d34ec1081903cbcb742589e995b4","size_bytes":6843},{"git_mode":"100644","path":"shared/utilities/yaml_loader.py","sha256":"203996ff297e1714b0a3e40446f719e24b6d0e20eddb91bc528100b497d8591c","size_bytes":679},{"git_mode":"100644","path":"shared/validation/__init__.py","sha256":"2c80d0a16e9b08c649b131deec3e32cfe9260d127c3b6e9c4a6508eb730e0e7f","size_bytes":1828},{"git_mode":"100644","path":"shared/validation/fitness.py","sha256":"6d90c803c59afa2e1054f33eac06c28a0d765160b53c090fd8b3bad4568cea48","size_bytes":8227},{"git_mode":"100644","path":"shared/validation/parsing/__init__.py","sha256":"e04b52ae2417aabdf8f48d80e847d60de41ec094824d4f6d088aaa132209b6c2","size_bytes":1768},{"git_mode":"100644","path":"shared/validation/parsing/configured_formats.py","sha256":"2d69eff733771bd614a8c1e1252511041fdd4bd5a13cf565205e1ebfb52dc88c","size_bytes":8162},{"git_mode":"100644","path":"shared/validation/parsing/enums.py","sha256":"4ea46d5ef556cc9d5d93d75f944e5a5120effea5264ccd89318d06228beb82bf","size_bytes":1746},{"git_mode":"100644","path":"shared/validation/parsing/response_parser.py","sha256":"5968f7d4f9f70ca269643276a71c238d6543778a47222724a1db69090b8c3471","size_bytes":21535},{"git_mode":"100644","path":"shared/validation/parsing/tool_call_parser.py","sha256":"e786758dd896bf75ceb49ea46a2a37fbea6d13692d7a1a63a26ae2872e33017a","size_bytes":13943},{"git_mode":"100644","path":"shared/validation/parsing/utilities.py","sha256":"42b1c7cb98807601a46066d5073e4d3e1984f4c848f632e46371b55e25732aa2","size_bytes":3024},{"git_mode":"100644","path":"shared/validation/rollout_filters.py","sha256":"d412c53eacb8e7fe9c459345e3dbf83a65ac50fe413d58d567d41a9c7fa6fc86","size_bytes":18795},{"git_mode":"100644","path":"shared/validation/rubric/__init__.py","sha256":"7df886540f94034002142322a481672a991d3059ec503cb996ec03e42d3123f6","size_bytes":438},{"git_mode":"100644","path":"shared/validation/rubric/rubric_cache.py","sha256":"d428466ebdadafe0c8cd8febab5e1fa49414d846efeb45c70cba47102532c2ad","size_bytes":3121},{"git_mode":"100644","path":"shared/validation/rubric/rubric_loader.py","sha256":"f77ca87d02885899d61db49fe709cba445e5c6d35fccdadb1deb8e052d8141e6","size_bytes":2313},{"git_mode":"100644","path":"shared/validation/rubric/rubric_repository.py","sha256":"1173978ad22f34625e981dc2148525c87c7745a927ee0d81c93bc1a030f56676","size_bytes":7104},{"git_mode":"100644","path":"shared/validation/validators/__init__.py","sha256":"d86436b05056c97a9b0aaae4465d23d03f407e737f4b2098fe3fc274fca64035","size_bytes":1031},{"git_mode":"100644","path":"shared/validation/validators/base.py","sha256":"764ae0d8a1439a4823a7ce82fafd1b69406873549c847b65ed57ecb3da97a264","size_bytes":1564},{"git_mode":"100644","path":"shared/validation/validators/content/__init__.py","sha256":"ed370a9fdc34b511dcfe88053bee6e9d4f30e3f7de2263cd3c675b8021865bb2","size_bytes":600},{"git_mode":"100644","path":"shared/validation/validators/content/code_validator.py","sha256":"19ded618181d16de6fa3667a64cc80a96628fd6d0f39ee5412f22c3e7aa75842","size_bytes":1493},{"git_mode":"100644","path":"shared/validation/validators/content/json_validator.py","sha256":"dd4b6fb4cb51aabaac37c052df9ff23b5051fb575dba2e893458d3d64edba497","size_bytes":7524},{"git_mode":"100644","path":"shared/validation/validators/content/regex_validator.py","sha256":"bea1124ccc79fec327a24a61e96110696c3346529d303ddff4a56ecba34b1513","size_bytes":3148},{"git_mode":"100644","path":"shared/validation/validators/content/xml_validator.py","sha256":"970937f6d8d9ef61594e2a34a2804d999482bf319c237a1891db214250bedd2e","size_bytes":1056},{"git_mode":"100644","path":"shared/validation/validators/content/yaml_validator.py","sha256":"75571df0e7ac750feeb8f92dc7da5bdc4d085e032bcf94650d826ecae07d3219","size_bytes":3738},{"git_mode":"100644","path":"shared/validation/validators/cross_scope_validator.py","sha256":"27ed2f25b96cc6fecd5e0702fdbe98a8248caf1483952a0d2402d96cf56ee269","size_bytes":8323},{"git_mode":"100644","path":"shared/validation/validators/structure_validator.py","sha256":"d637fde147e89d052229f3951ac744b56d87eb3f8553626b61dfd36bbb9829cf","size_bytes":20391},{"git_mode":"100644","path":"shared/verifiers/__init__.py","sha256":"bcc0a64a5b1be388e3dc1e913492506cac4daeb4cb5f7a210bca5bd75bafc352","size_bytes":950},{"git_mode":"100644","path":"shared/verifiers/builtins/__init__.py","sha256":"209700d4779871de25567fb6daeabad23ef3b8791e2f6c2b10749c2beead9c88","size_bytes":777},{"git_mode":"100644","path":"shared/verifiers/builtins/args_match.py","sha256":"19f44e00d8dc78d93e85361c9ecd65790fb6fa31f740f9aab743d36e20f38a0f","size_bytes":14581},{"git_mode":"100644","path":"shared/verifiers/builtins/assertion_verifier.py","sha256":"3ab2e6983adf1c9774e665961466856025fca6905d82094e2f0bc48e67cdfc46","size_bytes":17366},{"git_mode":"100644","path":"shared/verifiers/builtins/audio_verifier.py","sha256":"51c75365d411ab77535e343d771178f16fd6fee133b3d815eab0302c783fd207","size_bytes":16786},{"git_mode":"100644","path":"shared/verifiers/builtins/llm_judge.py","sha256":"b4741581e4388ed43bce5fd4def1465abc9b1eab3b8b4905d7ff07f05dd80d15","size_bytes":6915},{"git_mode":"100644","path":"shared/verifiers/builtins/retrieval_verifier.py","sha256":"b70c804369efa4a76dedeb1aeb829814952fecaecee51352803f100f12ac938d","size_bytes":15290},{"git_mode":"100644","path":"shared/verifiers/builtins/structure.py","sha256":"3030f1ca99e9355b0b37a8772937318ef334857aae5c05e92ad3db1318a4dc26","size_bytes":2167},{"git_mode":"100644","path":"shared/verifiers/builtins/substring.py","sha256":"dd86a419aa78058f5e855a538cf63e8a48c3e757f0988139bae0f231c531d76a","size_bytes":3790},{"git_mode":"100644","path":"shared/verifiers/builtins/tool_sequence.py","sha256":"111cb44e290877f1f265fd3d6d9e3d6635501b06f3648613af4f4f716b9f8d56","size_bytes":6459},{"git_mode":"100644","path":"shared/verifiers/composite.py","sha256":"678dbdb3bbab21f4a4956b7e70e8ef2d5dc883ec500a1d2278103a1f892ef66c","size_bytes":5681},{"git_mode":"100644","path":"shared/verifiers/contract.py","sha256":"c594c5527aa949d02cdf8626d03a5ef958148d647a7d7b84df6ee5c6b6ce56cc","size_bytes":2367},{"git_mode":"100644","path":"shared/verifiers/extraction.py","sha256":"f4a8e734957abb1327eeca7faa7cf5b1eb6d9247180224315e4ea6a42bc2e867","size_bytes":6056},{"git_mode":"100644","path":"shared/verifiers/registry.py","sha256":"f55035cd60ee539e4577ae1e4f101ed4b42d6c4adfb5bc0e17ae415607a3db2b","size_bytes":2475},{"git_mode":"100644","path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"git_mode":"100644","path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"git_mode":"100644","path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"git_mode":"100644","path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"git_mode":"100644","path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"git_mode":"100644","path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"git_mode":"100644","path":"synaptic_tuner/api/v1/observations.py","sha256":"b33f5826e6c03db889e0ff29c3eba65f5eab436b3d3f5146b8e4317828178d07","size_bytes":26569},{"git_mode":"100644","path":"synaptic_tuner/api/v1/planning.py","sha256":"e715b105a513fa56cfa510416e725471f8f3100cb34257be74bef3e40ca78460","size_bytes":9569},{"git_mode":"100644","path":"synaptic_tuner/api/v1/providers.py","sha256":"eca5efd5a6dee453543ae338ffddf9451d34ce1743e4ab4bde024b47be571ec0","size_bytes":4940},{"git_mode":"100644","path":"synaptic_tuner/api/v1/results.py","sha256":"65e530325c92c7afa18e3ca5f2ad9a11baf7d590221f08c33badfbf2d5f2c7b2","size_bytes":2236},{"git_mode":"100644","path":"synaptic_tuner/api/v1/training_facade.py","sha256":"bf094d0e152f5861bf8a8ff51962e0700123d9b43ed91d914b9e78bbea6facd9","size_bytes":11118},{"git_mode":"100644","path":"synaptic_tuner/api/v1/training_input.py","sha256":"209326325f8796ab7d038878e92def2c5521d7d98b7b9cc4dd1766d2e4f68c9d","size_bytes":38650},{"git_mode":"100644","path":"synaptic_tuner/api/v1/training_sources.py","sha256":"4c4ce3bc9d73e03d550c61f5ef8f18099137fe82a9be6ea96b226e7363e984b8","size_bytes":8454},{"git_mode":"100644","path":"synaptic_tuner/api/v1/usage.py","sha256":"33e36965e14ae52037f1da877c6606c46a40edf575e248769d5e5731b47b1307","size_bytes":6086},{"git_mode":"100644","path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"git_mode":"100644","path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"git_mode":"100644","path":"tuner/cloud/runtime_layout.py","sha256":"2a4cd1f5fc63d95bb3f0b5bc4f44e616bb0308d9509aceb198943953746ff705","size_bytes":9435},{"git_mode":"100644","path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"git_mode":"100644","path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"git_mode":"100644","path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"git_mode":"100644","path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"git_mode":"100644","path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"git_mode":"100644","path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"git_mode":"100644","path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"git_mode":"100644","path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"git_mode":"100644","path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"git_mode":"100644","path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"git_mode":"100644","path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"git_mode":"100644","path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"git_mode":"100644","path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"git_mode":"100644","path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"git_mode":"100644","path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"git_mode":"100644","path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"git_mode":"100644","path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"git_mode":"100644","path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"git_mode":"100644","path":"tuner/runtime/dispatch.py","sha256":"4c41faf87d7e620bdc281490bf3725550b83dacca8b15ea47bab253f669cdd8e","size_bytes":35463},{"git_mode":"100644","path":"tuner/runtime/offline_grpo_worker.py","sha256":"aac831048855e959598a4dc45b29e614f06f8714df18e868fd6b069e3c765710","size_bytes":9323},{"git_mode":"100644","path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_child.py","sha256":"124d5b2a969d38ac2e488d58a216759108f1a55dfcc3dcaf1012791efe792e4b","size_bytes":8273},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_execution.py","sha256":"a77c7d6b97ad2ded3b9631174e479e60462f1894513f90fd2fc7798828c6867d","size_bytes":14943},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_verification.py","sha256":"56564c6ee6414a3a71ce7c9ef330152d812fede2a711a2329463a0b81a61b0d6","size_bytes":10319},{"git_mode":"100644","path":"tuner/runtime/packaged_grpo_worker_closure.py","sha256":"81da8db5a61e9a874234d1e50c529e1056282fd0eef133b2720eb0f34f78cddc","size_bytes":3735},{"git_mode":"100644","path":"tuner/runtime/packaged_sft_child.py","sha256":"ab41eb7dca28b440c043f589d297dc0d9f718ed14f6cb36c96d7bd1b7bfcdc14","size_bytes":23556},{"git_mode":"100644","path":"tuner/runtime/packaged_sft_execution.py","sha256":"cab2b5032d16146422398305120f52c5a124287ecf929b640a1f17d0ea2ed648","size_bytes":80053},{"git_mode":"100644","path":"tuner/runtime/packaged_training_worker.py","sha256":"9cfe13dc9f0124b4da01d05e92cf3c2c9249d075c3a508d6b93b0a7c7f0e109f","size_bytes":34986},{"git_mode":"100644","path":"tuner/runtime/packaged_worker_closure.py","sha256":"e457937426e19747e867a314c1a2556ee64bd5cbc19359f535253b02403d1443","size_bytes":8705},{"git_mode":"100644","path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"git_mode":"100644","path":"tuner/runtime/verification.py","sha256":"4c2314b20cbf7937a3f24aa90676000b9e21f0ab6701b2c1225a53e5596bf4a0","size_bytes":53856},{"git_mode":"100644","path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"git_mode":"100644","path":"tuner/training/contracts.py","sha256":"ebe52af8ba1c02309d39317694bc04547a6491d110c60229282517dbdab5a417","size_bytes":21524},{"git_mode":"100644","path":"tuner/training/grpo_compilation.py","sha256":"c6268486375bbc16f1888bb43d07b93a8de280d000a774914d59d7fd05ca5cc0","size_bytes":8998},{"git_mode":"100644","path":"tuner/training/packaged_compilation.py","sha256":"477caede6902fc587af680889b93fd9e0e42493d1a6653f3cd7413a9c7cadde8","size_bytes":9413},{"git_mode":"100644","path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722},{"git_mode":"100644","path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"optional_features":[],"owned_module_prefixes":["tuner","synaptic_tuner","Trainers","shared","SynthChat","Evaluator","MechInterp","configs","src"],"payload_bytes":1408467,"schema_version":"synaptic-offline-grpo-worker-closure/v1","trainer_entrypoint":"Trainers/grpo/runtime_v1.py"} diff --git a/tuner/runtime/manifests/offline-sft-worker-v1.json b/tuner/runtime/manifests/offline-sft-worker-v1.json index ed73ccfa..6bbd57e7 100644 --- a/tuner/runtime/manifests/offline-sft-worker-v1.json +++ b/tuner/runtime/manifests/offline-sft-worker-v1.json @@ -1 +1 @@ -{"closure_digest":"8465ff2b7cedb352337e7cedc5bb17cc169892c2eedae0b280569f4a0cde5002","closure_ref":"synaptic/offline-sft-worker/v1","entrypoint":"Trainers/sft/runtime_v1.py","member_count":68,"members":[{"git_mode":"100644","path":"Trainers/sft/configs/config.yaml","sha256":"93b85493fa4200a103267ca7ba1223dcd519be3d0cf9017304ebcdc2e7a99ec1","size_bytes":5422},{"git_mode":"100644","path":"Trainers/sft/configs/config_loader.py","sha256":"4f90bf152d191544f9a567e22f41a171aa199e8995a6a7ea00a7098bc44a4060","size_bytes":19126},{"git_mode":"100644","path":"Trainers/sft/runtime_v1.py","sha256":"460121aaee3eb83764f67d9307cfb57a3b25c117611d987577b9631b6956a1b3","size_bytes":104495},{"git_mode":"100644","path":"Trainers/sft/src/data_loader.py","sha256":"34d664535b4d26f9a6e9b1a4121e37b575f3960ea39b6fd651f2fe9b624ffb09","size_bytes":19916},{"git_mode":"100644","path":"Trainers/sft/src/model_loader.py","sha256":"00322b6e586eb0245675d1a5245172409ec477bbaccf33bf54af8534d9252d2b","size_bytes":19824},{"git_mode":"100644","path":"Trainers/sft/src/preprocessing.py","sha256":"db52901528b4a48d0a3ae3afbf2ec664b5e957ed467161a8535b0d953588a40e","size_bytes":15145},{"git_mode":"100644","path":"Trainers/sft/src/training_callbacks.py","sha256":"d04ed8c3d95361a2b73443de16519dc9540c96ee4276f80b5d109eaf13cd7b98","size_bytes":4201},{"git_mode":"100644","path":"Trainers/sft/train_sft.py","sha256":"75c2ab22fbc5dd80b8ddd5e6d0da816d1451cda0cd620c34717336a39a8a7644","size_bytes":88561},{"git_mode":"100644","path":"Trainers/shared/__init__.py","sha256":"39bf91bda8d3a2a76f70275c67b66d28f2bdd0c53c4d38b96f20ef540e431b6d","size_bytes":217},{"git_mode":"100644","path":"Trainers/shared/callbacks/__init__.py","sha256":"a919bb7953721dd2e001cad1723c5a8ba9c4e77608505df1570fba8906a3ff33","size_bytes":1743},{"git_mode":"100644","path":"Trainers/shared/callbacks/base.py","sha256":"4c8225571ae38b321dce1898f44fba62393d95e5adbcf072ad69007e937ec680","size_bytes":25232},{"git_mode":"100644","path":"Trainers/shared/callbacks/checkpoints.py","sha256":"3b86700a42dc0cc2d37a9fbebe5c5f97fc468bed310eecb6c5d76f3d2aff0080","size_bytes":561},{"git_mode":"100644","path":"Trainers/shared/callbacks/health_checks.py","sha256":"9f3732cee2173ada04a19c1e626100ca6ed1a50c12cb9809f48cc7cc004ee14f","size_bytes":3388},{"git_mode":"100644","path":"Trainers/shared/callbacks/log_suppression.py","sha256":"b30cc981f9f3c4f2046afdbf294ac848914f3757743f4d5d11a3280296aecaa2","size_bytes":787},{"git_mode":"100644","path":"Trainers/shared/callbacks/lr_schedules.py","sha256":"8e66ce42956fd7eedaa8dbe0a91feeb26c39df0cb0faf4871a1022a0aa7b6623","size_bytes":1938},{"git_mode":"100644","path":"schemas/synaptic-execution-source-v1.schema.json","sha256":"fa80f663b64b4ffdfb2e54c0a321cf7374ebeeb850b56b7a2fa00fd51ff81455","size_bytes":6227},{"git_mode":"100644","path":"schemas/synaptic-sft-workload-v1.schema.json","sha256":"d6d0c756619fc58f10041ff73e205cab99a14435e17379d950757dfbde4d698e","size_bytes":6651},{"git_mode":"100644","path":"shared/cloud_artifacts.py","sha256":"c7f95ef077b0bb5102ac66b6e4fd80b07470edec5e3fefd0d8fffaa6238eca4a","size_bytes":16737},{"git_mode":"100644","path":"shared/env_bootstrap.py","sha256":"4a8d3dd248639ea96fb8f94e5587b190145e49f75d2ebe2a01f05e4e1521c85c","size_bytes":4344},{"git_mode":"100644","path":"shared/experiment_tracking/__init__.py","sha256":"96e521ab64e8b7697d3aa7368346db7b8215997ffe4604cadaa8b022e39a5db2","size_bytes":2305},{"git_mode":"100644","path":"shared/experiment_tracking/experiment.py","sha256":"eb569f16b8b37f63ed6ba0ee7eaa76daf6940923c8c0c5a63b84a659419dae00","size_bytes":18628},{"git_mode":"100644","path":"shared/experiment_tracking/experiment_spec.py","sha256":"2ab5aaf808b49bdccc99cd857394ac4898e5242af33023a9855c702181394da3","size_bytes":11670},{"git_mode":"100644","path":"shared/experiment_tracking/lineage_enrichment.py","sha256":"accd4bb5b46af302b983c4c49b0f13976841d8b12672149feb98f99886fc3be5","size_bytes":9075},{"git_mode":"100644","path":"shared/experiment_tracking/schema.py","sha256":"871a00a4d4c90030a29be2d5c94669bed244921f00898c750423c29a92e02465","size_bytes":18789},{"git_mode":"100644","path":"shared/sft_preprocessing.py","sha256":"3cbac1e9b3d75fe6d7f1d6cb578d8c8b05de669249526a6f92a41484e4cb42b2","size_bytes":33273},{"git_mode":"100644","path":"shared/training_capacity.py","sha256":"2fae8c83e423a66b0ec56abc0e415363d790c12329120aac4c2103c298ad0e48","size_bytes":19299},{"git_mode":"100644","path":"shared/training_utils.py","sha256":"15136abdb987a7bdfbc8917c49f9c77ff553e91ce53890e239ea629241345a65","size_bytes":28814},{"git_mode":"100644","path":"shared/utilities/__init__.py","sha256":"546349addf2ba6c72661e185477354d743f6a42e8ef3bf8e78ed19a17b85e2ab","size_bytes":297},{"git_mode":"100644","path":"shared/utilities/env.py","sha256":"06f56639bc506d4e497ee79d4393d17d499bfd561fd0d0679c0ea7e103b8e886","size_bytes":3318},{"git_mode":"100644","path":"shared/utilities/paths.py","sha256":"3bfa1a13e4deef02cd91a920410e839f8ab0d34ec1081903cbcb742589e995b4","size_bytes":6843},{"git_mode":"100644","path":"shared/utilities/unique_ids.py","sha256":"4f5fd65acc46fc7458a79047403a3f50d5a92c96ea6efcd7ffc560aa329e4fae","size_bytes":678},{"git_mode":"100644","path":"shared/utilities/yaml_loader.py","sha256":"203996ff297e1714b0a3e40446f719e24b6d0e20eddb91bc528100b497d8591c","size_bytes":679},{"git_mode":"100644","path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"git_mode":"100644","path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"git_mode":"100644","path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"git_mode":"100644","path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"git_mode":"100644","path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"git_mode":"100644","path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"git_mode":"100644","path":"synaptic_tuner/api/v1/execution.py","sha256":"3309cbfdea5a7746b0463147a22bb61d8e14028134ef5a2b05dadcad985fdfbc","size_bytes":5506},{"git_mode":"100644","path":"synaptic_tuner/api/v1/observations.py","sha256":"b33f5826e6c03db889e0ff29c3eba65f5eab436b3d3f5146b8e4317828178d07","size_bytes":26569},{"git_mode":"100644","path":"synaptic_tuner/api/v1/sources.py","sha256":"65e410cdfe331760bbb03626cd58284d0a5f946b4cc081ba457ad639f4c1e5ef","size_bytes":858},{"git_mode":"100644","path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"git_mode":"100644","path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"git_mode":"100644","path":"tuner/cloud/hardware_planner.py","sha256":"62467f775f5201229e5dc0f34ed72caed760724466dcf990a9c3fb9e7ca04c5d","size_bytes":25209},{"git_mode":"100644","path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"git_mode":"100644","path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"git_mode":"100644","path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"git_mode":"100644","path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"git_mode":"100644","path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"git_mode":"100644","path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"git_mode":"100644","path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"git_mode":"100644","path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"git_mode":"100644","path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"git_mode":"100644","path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"git_mode":"100644","path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"git_mode":"100644","path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"git_mode":"100644","path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"git_mode":"100644","path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"git_mode":"100644","path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"git_mode":"100644","path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"git_mode":"100644","path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"git_mode":"100644","path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"git_mode":"100644","path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"git_mode":"100644","path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"git_mode":"100644","path":"tuner/training/contracts.py","sha256":"91c4d66d9810cc0471981c3d4ef4b54feaed5b92eb41a468f31201f8111303b5","size_bytes":21504},{"git_mode":"100644","path":"tuner/training/methods/__init__.py","sha256":"99db689246273707ae3df5087859ebb510e84b664f9f173c0d5827abc3e6a3ea","size_bytes":361},{"git_mode":"100644","path":"tuner/training/methods/sft.py","sha256":"b501f9c214dec4b0b1550543281c25aba8dbb102604df253c641c85f7cae587b","size_bytes":9997},{"git_mode":"100644","path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"optional_features":[],"owned_module_prefixes":["tuner","synaptic_tuner","Trainers","shared","SynthChat","Evaluator","MechInterp","configs","src"],"payload_bytes":842015,"schema_version":"synaptic-offline-sft-worker-closure/v1","trainer_entrypoint":"Trainers/sft/train_sft.py"} +{"closure_digest":"2fb55c7ab83c91d132b23eff99cd4a2e056b446436dced0c1a54b8da122fbd7e","closure_ref":"synaptic/offline-sft-worker/v1","entrypoint":"Trainers/sft/runtime_v1.py","member_count":68,"members":[{"git_mode":"100644","path":"Trainers/sft/configs/config.yaml","sha256":"93b85493fa4200a103267ca7ba1223dcd519be3d0cf9017304ebcdc2e7a99ec1","size_bytes":5422},{"git_mode":"100644","path":"Trainers/sft/configs/config_loader.py","sha256":"4f90bf152d191544f9a567e22f41a171aa199e8995a6a7ea00a7098bc44a4060","size_bytes":19126},{"git_mode":"100644","path":"Trainers/sft/runtime_v1.py","sha256":"460121aaee3eb83764f67d9307cfb57a3b25c117611d987577b9631b6956a1b3","size_bytes":104495},{"git_mode":"100644","path":"Trainers/sft/src/data_loader.py","sha256":"34d664535b4d26f9a6e9b1a4121e37b575f3960ea39b6fd651f2fe9b624ffb09","size_bytes":19916},{"git_mode":"100644","path":"Trainers/sft/src/model_loader.py","sha256":"00322b6e586eb0245675d1a5245172409ec477bbaccf33bf54af8534d9252d2b","size_bytes":19824},{"git_mode":"100644","path":"Trainers/sft/src/preprocessing.py","sha256":"db52901528b4a48d0a3ae3afbf2ec664b5e957ed467161a8535b0d953588a40e","size_bytes":15145},{"git_mode":"100644","path":"Trainers/sft/src/training_callbacks.py","sha256":"d04ed8c3d95361a2b73443de16519dc9540c96ee4276f80b5d109eaf13cd7b98","size_bytes":4201},{"git_mode":"100644","path":"Trainers/sft/train_sft.py","sha256":"75c2ab22fbc5dd80b8ddd5e6d0da816d1451cda0cd620c34717336a39a8a7644","size_bytes":88561},{"git_mode":"100644","path":"Trainers/shared/__init__.py","sha256":"39bf91bda8d3a2a76f70275c67b66d28f2bdd0c53c4d38b96f20ef540e431b6d","size_bytes":217},{"git_mode":"100644","path":"Trainers/shared/callbacks/__init__.py","sha256":"a919bb7953721dd2e001cad1723c5a8ba9c4e77608505df1570fba8906a3ff33","size_bytes":1743},{"git_mode":"100644","path":"Trainers/shared/callbacks/base.py","sha256":"4c8225571ae38b321dce1898f44fba62393d95e5adbcf072ad69007e937ec680","size_bytes":25232},{"git_mode":"100644","path":"Trainers/shared/callbacks/checkpoints.py","sha256":"3b86700a42dc0cc2d37a9fbebe5c5f97fc468bed310eecb6c5d76f3d2aff0080","size_bytes":561},{"git_mode":"100644","path":"Trainers/shared/callbacks/health_checks.py","sha256":"9f3732cee2173ada04a19c1e626100ca6ed1a50c12cb9809f48cc7cc004ee14f","size_bytes":3388},{"git_mode":"100644","path":"Trainers/shared/callbacks/log_suppression.py","sha256":"b30cc981f9f3c4f2046afdbf294ac848914f3757743f4d5d11a3280296aecaa2","size_bytes":787},{"git_mode":"100644","path":"Trainers/shared/callbacks/lr_schedules.py","sha256":"8e66ce42956fd7eedaa8dbe0a91feeb26c39df0cb0faf4871a1022a0aa7b6623","size_bytes":1938},{"git_mode":"100644","path":"schemas/synaptic-execution-source-v1.schema.json","sha256":"fa80f663b64b4ffdfb2e54c0a321cf7374ebeeb850b56b7a2fa00fd51ff81455","size_bytes":6227},{"git_mode":"100644","path":"schemas/synaptic-sft-workload-v1.schema.json","sha256":"d6d0c756619fc58f10041ff73e205cab99a14435e17379d950757dfbde4d698e","size_bytes":6651},{"git_mode":"100644","path":"shared/cloud_artifacts.py","sha256":"c7f95ef077b0bb5102ac66b6e4fd80b07470edec5e3fefd0d8fffaa6238eca4a","size_bytes":16737},{"git_mode":"100644","path":"shared/env_bootstrap.py","sha256":"4a8d3dd248639ea96fb8f94e5587b190145e49f75d2ebe2a01f05e4e1521c85c","size_bytes":4344},{"git_mode":"100644","path":"shared/experiment_tracking/__init__.py","sha256":"96e521ab64e8b7697d3aa7368346db7b8215997ffe4604cadaa8b022e39a5db2","size_bytes":2305},{"git_mode":"100644","path":"shared/experiment_tracking/experiment.py","sha256":"eb569f16b8b37f63ed6ba0ee7eaa76daf6940923c8c0c5a63b84a659419dae00","size_bytes":18628},{"git_mode":"100644","path":"shared/experiment_tracking/experiment_spec.py","sha256":"2ab5aaf808b49bdccc99cd857394ac4898e5242af33023a9855c702181394da3","size_bytes":11670},{"git_mode":"100644","path":"shared/experiment_tracking/lineage_enrichment.py","sha256":"accd4bb5b46af302b983c4c49b0f13976841d8b12672149feb98f99886fc3be5","size_bytes":9075},{"git_mode":"100644","path":"shared/experiment_tracking/schema.py","sha256":"871a00a4d4c90030a29be2d5c94669bed244921f00898c750423c29a92e02465","size_bytes":18789},{"git_mode":"100644","path":"shared/sft_preprocessing.py","sha256":"3cbac1e9b3d75fe6d7f1d6cb578d8c8b05de669249526a6f92a41484e4cb42b2","size_bytes":33273},{"git_mode":"100644","path":"shared/training_capacity.py","sha256":"2fae8c83e423a66b0ec56abc0e415363d790c12329120aac4c2103c298ad0e48","size_bytes":19299},{"git_mode":"100644","path":"shared/training_utils.py","sha256":"15136abdb987a7bdfbc8917c49f9c77ff553e91ce53890e239ea629241345a65","size_bytes":28814},{"git_mode":"100644","path":"shared/utilities/__init__.py","sha256":"546349addf2ba6c72661e185477354d743f6a42e8ef3bf8e78ed19a17b85e2ab","size_bytes":297},{"git_mode":"100644","path":"shared/utilities/env.py","sha256":"06f56639bc506d4e497ee79d4393d17d499bfd561fd0d0679c0ea7e103b8e886","size_bytes":3318},{"git_mode":"100644","path":"shared/utilities/paths.py","sha256":"3bfa1a13e4deef02cd91a920410e839f8ab0d34ec1081903cbcb742589e995b4","size_bytes":6843},{"git_mode":"100644","path":"shared/utilities/unique_ids.py","sha256":"4f5fd65acc46fc7458a79047403a3f50d5a92c96ea6efcd7ffc560aa329e4fae","size_bytes":678},{"git_mode":"100644","path":"shared/utilities/yaml_loader.py","sha256":"203996ff297e1714b0a3e40446f719e24b6d0e20eddb91bc528100b497d8591c","size_bytes":679},{"git_mode":"100644","path":"synaptic_tuner/__init__.py","sha256":"2462ff4ec439b77eb1ecfa901c1d8294628c853e9bbdf872de73dce9bfc0ae4d","size_bytes":134},{"git_mode":"100644","path":"synaptic_tuner/_version.py","sha256":"49fa86b62849d537e7e131445f272650ed77760db5f218f2113155ab60caf644","size_bytes":96},{"git_mode":"100644","path":"synaptic_tuner/api/__init__.py","sha256":"25c68ed273bf1e6b158021b98f8607276d980540a98b56a07801dccf9d6596a8","size_bytes":67},{"git_mode":"100644","path":"synaptic_tuner/api/v1/__init__.py","sha256":"4bc30420b18b3932183dcd0485892fa22f4f8e474ca3bb9b5e08adf3ae0fde3d","size_bytes":16893},{"git_mode":"100644","path":"synaptic_tuner/api/v1/_contract.py","sha256":"dcd6fc1ebdc75fbfd74ad4c6c96837b73209081258353557fedaaca76164673a","size_bytes":6702},{"git_mode":"100644","path":"synaptic_tuner/api/v1/_timestamps.py","sha256":"2b2ad7183f62bf1d42c4b9144d0315fe90c0e601649fb37f7778d017c632935d","size_bytes":698},{"git_mode":"100644","path":"synaptic_tuner/api/v1/execution.py","sha256":"3309cbfdea5a7746b0463147a22bb61d8e14028134ef5a2b05dadcad985fdfbc","size_bytes":5506},{"git_mode":"100644","path":"synaptic_tuner/api/v1/observations.py","sha256":"b33f5826e6c03db889e0ff29c3eba65f5eab436b3d3f5146b8e4317828178d07","size_bytes":26569},{"git_mode":"100644","path":"synaptic_tuner/api/v1/sources.py","sha256":"65e410cdfe331760bbb03626cd58284d0a5f946b4cc081ba457ad639f4c1e5ef","size_bytes":858},{"git_mode":"100644","path":"tuner/__init__.py","sha256":"920cf8a0ca993a712fbacf734621e1fdf14fdd2a7203701db120dc41e4815e7b","size_bytes":2666},{"git_mode":"100644","path":"tuner/cloud/__init__.py","sha256":"a9abb71aa9222c511524d3cc49b1cfe00f8938a6cb7c61792cf6135a6b82208b","size_bytes":3692},{"git_mode":"100644","path":"tuner/cloud/hardware_planner.py","sha256":"62467f775f5201229e5dc0f34ed72caed760724466dcf990a9c3fb9e7ca04c5d","size_bytes":25209},{"git_mode":"100644","path":"tuner/execution/__init__.py","sha256":"8193dfcd30c68a7d8ad02af0fb24f1a8120fce2a709decd132ab1acb33113d65","size_bytes":535},{"git_mode":"100644","path":"tuner/execution/contracts.py","sha256":"82d90f742e4e33d8071e557b5a872e0158684d94db5debdfbb85411954e4e638","size_bytes":32907},{"git_mode":"100644","path":"tuner/execution/evidence.py","sha256":"a04cc6843a794e62a702608d28c8e82d0c57af8227b3cb9743cf56b9605442ca","size_bytes":4634},{"git_mode":"100644","path":"tuner/execution/lifecycle.py","sha256":"4bc878038fc2384c0213a5f5c887aca8b15a67ed46b11e1acae1e2ecf0e97793","size_bytes":17660},{"git_mode":"100644","path":"tuner/execution/registry.py","sha256":"0cf424ff4b6bd628bae86b347f00a10acce8793cc41afc2ff1f3abbb738aec40","size_bytes":1515},{"git_mode":"100644","path":"tuner/execution/service.py","sha256":"a85e31e8a54479764463633a9c721b9af3afe38710f85efa35b9a5744ad33e4c","size_bytes":4514},{"git_mode":"100644","path":"tuner/project/__init__.py","sha256":"a858562f45452031f8a0e8ba4c9799f9784e09360a0258165dc3c4f89a46ca23","size_bytes":1513},{"git_mode":"100644","path":"tuner/project/config_layers.py","sha256":"0e06ddfb8b92faa106d052990df34f199c06c298c3a3a2366cab218f2c1731f1","size_bytes":6714},{"git_mode":"100644","path":"tuner/project/context.py","sha256":"147d08d5d6fcb3cf258ab347f070424e65e5c0c28e47ad1fd062e13cf56a8dd3","size_bytes":6518},{"git_mode":"100644","path":"tuner/project/errors.py","sha256":"6f1f94cd0b4e068a0cdee82259de0aee09353345e93caeb05272097a5e717444","size_bytes":1511},{"git_mode":"100644","path":"tuner/project/execution_source.py","sha256":"dc196dff10029c9a77b6d94fbfbcca45b39da4840bff40009b7152fd39d9622b","size_bytes":30595},{"git_mode":"100644","path":"tuner/project/git_verification.py","sha256":"0cfe00560c704eddbe688a0c7156c12f9acd2f321046017f50eb5d9e3bb61710","size_bytes":9767},{"git_mode":"100644","path":"tuner/project/manifest.py","sha256":"5506cf2ffb261988c0bbc5ffab1e466236bd10f503df5e805453f5ddf7d34be0","size_bytes":9498},{"git_mode":"100644","path":"tuner/project/path_refs.py","sha256":"35e47de6ed1c6096ba45e7ea64aedcc0f79aeb9425acb5041b14afb9d258c955","size_bytes":6287},{"git_mode":"100644","path":"tuner/project/secrets.py","sha256":"6f267060b0f24ab1517c4d4499fe54a993a77c2c78830c2d79415da871893818","size_bytes":4256},{"git_mode":"100644","path":"tuner/project/source_bundle.py","sha256":"16368658db727680f74326603e2d28f3ae9b5e481f7626c7be1a8280b588ecec","size_bytes":28339},{"git_mode":"100644","path":"tuner/runtime/__init__.py","sha256":"246429ce58a07dabeb05bf17967d3a7862014d114ee995a7419adcb27227286c","size_bytes":3566},{"git_mode":"100644","path":"tuner/runtime/artifacts.py","sha256":"c9eaefd6f6805af11d997256e14392c0c42173b852d94bfab90bcb20a1ae2cd2","size_bytes":6913},{"git_mode":"100644","path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"git_mode":"100644","path":"tuner/training/__init__.py","sha256":"23ca59d6203276c6160e0bb57dbd5c43cf385c6aec60b4e3e08b93d1d501cb97","size_bytes":2995},{"git_mode":"100644","path":"tuner/training/contracts.py","sha256":"ebe52af8ba1c02309d39317694bc04547a6491d110c60229282517dbdab5a417","size_bytes":21524},{"git_mode":"100644","path":"tuner/training/methods/__init__.py","sha256":"99db689246273707ae3df5087859ebb510e84b664f9f173c0d5827abc3e6a3ea","size_bytes":361},{"git_mode":"100644","path":"tuner/training/methods/sft.py","sha256":"b501f9c214dec4b0b1550543281c25aba8dbb102604df253c641c85f7cae587b","size_bytes":9997},{"git_mode":"100644","path":"tuner/training/recipes.py","sha256":"bb844ed4883f00ccea9936eebad50c466e6e4bc148e88aa680d93121edf67dcc","size_bytes":6300}],"optional_features":[],"owned_module_prefixes":["tuner","synaptic_tuner","Trainers","shared","SynthChat","Evaluator","MechInterp","configs","src"],"payload_bytes":842035,"schema_version":"synaptic-offline-sft-worker-closure/v1","trainer_entrypoint":"Trainers/sft/train_sft.py"} diff --git a/tuner/runtime/manifests/packaged-grpo-worker-v1.json b/tuner/runtime/manifests/packaged-grpo-worker-v1.json index 2f1dac12..84706154 100644 --- a/tuner/runtime/manifests/packaged-grpo-worker-v1.json +++ b/tuner/runtime/manifests/packaged-grpo-worker-v1.json @@ -1 +1 @@ -{"closure_digest":"e48763871e7e9bc50c0c2b218bbfcb885b72b56f2ae1c2c41b1e16938c198ffe","members":[{"path":"tuner/runtime/manifests/offline-grpo-worker-v1.json","sha256":"7159247a2d15faaf2654b14168c1f90359b09950f646d0e9b2e2ec683f20617b","size_bytes":21344},{"path":"tuner/runtime/offline_grpo_worker.py","sha256":"aac831048855e959598a4dc45b29e614f06f8714df18e868fd6b069e3c765710","size_bytes":9323},{"path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"path":"tuner/runtime/packaged_grpo_child.py","sha256":"124d5b2a969d38ac2e488d58a216759108f1a55dfcc3dcaf1012791efe792e4b","size_bytes":8273},{"path":"tuner/runtime/packaged_grpo_execution.py","sha256":"a77c7d6b97ad2ded3b9631174e479e60462f1894513f90fd2fc7798828c6867d","size_bytes":14943},{"path":"tuner/runtime/packaged_grpo_verification.py","sha256":"56564c6ee6414a3a71ce7c9ef330152d812fede2a711a2329463a0b81a61b0d6","size_bytes":10319},{"path":"tuner/runtime/packaged_grpo_worker_closure.py","sha256":"81da8db5a61e9a874234d1e50c529e1056282fd0eef133b2720eb0f34f78cddc","size_bytes":3735},{"path":"tuner/runtime/packaged_sft_child.py","sha256":"ab41eb7dca28b440c043f589d297dc0d9f718ed14f6cb36c96d7bd1b7bfcdc14","size_bytes":23556},{"path":"tuner/runtime/packaged_sft_execution.py","sha256":"cab2b5032d16146422398305120f52c5a124287ecf929b640a1f17d0ea2ed648","size_bytes":80053},{"path":"tuner/runtime/packaged_training_worker.py","sha256":"9cfe13dc9f0124b4da01d05e92cf3c2c9249d075c3a508d6b93b0a7c7f0e109f","size_bytes":34986},{"path":"tuner/runtime/packaged_worker_closure.py","sha256":"e457937426e19747e867a314c1a2556ee64bd5cbc19359f535253b02403d1443","size_bytes":8705},{"path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722}],"schema_version":"synaptic-packaged-grpo-worker-closure/v1"} +{"closure_digest":"ad135893a01d729f38f1a769c02856ee3b5688ef60c3653f811f0317e1f7f53c","members":[{"path":"tuner/runtime/manifests/offline-grpo-worker-v1.json","sha256":"76390b7a5d9eaa419146bcb5c2ecd6493dccc23391395ccd9d72b782ea3787aa","size_bytes":21344},{"path":"tuner/runtime/offline_grpo_worker.py","sha256":"aac831048855e959598a4dc45b29e614f06f8714df18e868fd6b069e3c765710","size_bytes":9323},{"path":"tuner/runtime/offline_sft_worker.py","sha256":"2304eeb00667954510c34cc4aacaa818532a301fbeabf04c2fb165d6cd1d70b5","size_bytes":36343},{"path":"tuner/runtime/packaged_grpo_child.py","sha256":"124d5b2a969d38ac2e488d58a216759108f1a55dfcc3dcaf1012791efe792e4b","size_bytes":8273},{"path":"tuner/runtime/packaged_grpo_execution.py","sha256":"a77c7d6b97ad2ded3b9631174e479e60462f1894513f90fd2fc7798828c6867d","size_bytes":14943},{"path":"tuner/runtime/packaged_grpo_verification.py","sha256":"56564c6ee6414a3a71ce7c9ef330152d812fede2a711a2329463a0b81a61b0d6","size_bytes":10319},{"path":"tuner/runtime/packaged_grpo_worker_closure.py","sha256":"81da8db5a61e9a874234d1e50c529e1056282fd0eef133b2720eb0f34f78cddc","size_bytes":3735},{"path":"tuner/runtime/packaged_sft_child.py","sha256":"ab41eb7dca28b440c043f589d297dc0d9f718ed14f6cb36c96d7bd1b7bfcdc14","size_bytes":23556},{"path":"tuner/runtime/packaged_sft_execution.py","sha256":"cab2b5032d16146422398305120f52c5a124287ecf929b640a1f17d0ea2ed648","size_bytes":80053},{"path":"tuner/runtime/packaged_training_worker.py","sha256":"9cfe13dc9f0124b4da01d05e92cf3c2c9249d075c3a508d6b93b0a7c7f0e109f","size_bytes":34986},{"path":"tuner/runtime/packaged_worker_closure.py","sha256":"e457937426e19747e867a314c1a2556ee64bd5cbc19359f535253b02403d1443","size_bytes":8705},{"path":"tuner/runtime/releases.py","sha256":"c0aa7ba37e8d35287fdd480cf0c76053a554e546c40c8bf36f5cf05d1a96d22b","size_bytes":52535},{"path":"tuner/training/packaged_grpo_compilation.py","sha256":"d186a05ffb8cc756e782b2bb9a3f66e925779104b70045bd56a5592846fec524","size_bytes":8722}],"schema_version":"synaptic-packaged-grpo-worker-closure/v1"} diff --git a/tuner/runtime/native_files.py b/tuner/runtime/native_files.py new file mode 100644 index 00000000..f5773afa --- /dev/null +++ b/tuner/runtime/native_files.py @@ -0,0 +1,217 @@ +"""Bounded POSIX file reads for native macOS candidates. + +Uses openat/dir_fd and O_NOFOLLOW, available on supported Darwin and Linux. +There is no /proc, openat2, pidfd, or claim of protection from a malicious +process running as the same uid. The host retains exclusive mutation ownership. +""" + +from __future__ import annotations + +from contextlib import ExitStack +from dataclasses import dataclass +import hashlib +import os +from pathlib import Path, PurePosixPath +import stat + + +def file_identity(info): + return (info.st_dev, info.st_ino, info.st_size, info.st_mtime_ns, info.st_ctime_ns) + + +def relative_member(value: str) -> tuple[str, ...]: + if type(value) is not str or not value or len(value.encode()) > 4096: + raise ValueError("native member path is invalid") + path = PurePosixPath(value) + if path.is_absolute() or str(path) != value or any(p in (".", "..") for p in path.parts): + raise ValueError("native member path is noncanonical") + if any(ord(c) < 32 for c in value) or "\\" in value: + raise ValueError("native member path is invalid") + return path.parts + + +@dataclass(frozen=True) +class NativeFile: + path: str + size: int + sha256: str + + def __post_init__(self): + relative_member(self.path) + if type(self.size) is not int or not 0 <= self.size <= 512 * 1024 * 1024: + raise ValueError("native file size is invalid") + if (type(self.sha256) is not str or len(self.sha256) != 64 + or any(c not in "0123456789abcdef" for c in self.sha256)): + raise ValueError("native file digest is invalid") + + +class NativeTree: + """Retain all ancestors; reject redirected, non-owned or writable roots. + + Ordinary runtime trees may have mode 0755. Output/input private trees must + have no group/other permission. An OS-owned ancestor is allowed, but the + selected tree itself must belong to this uid. + """ + + def __init__(self, path: Path, *, private: bool = False): + if os.name != "posix" or not hasattr(os, "O_NOFOLLOW"): + raise ValueError("native descriptor traversal is unavailable") + if not path.is_absolute() or str(path) != os.path.normpath(str(path)): + raise ValueError("native root must be canonical absolute") + self.path = path + self._private = private + self._stack = ExitStack() + self._chain = [] + try: + parent = None + for part in ("/", *path.parts[1:]): + fd = os.open(part, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW, dir_fd=parent) + self._stack.callback(os.close, fd) + info = os.fstat(fd) + if info.st_uid not in (0, os.getuid()) or info.st_mode & 0o022: + # /tmp is intentionally not accepted as a native runtime root. + raise ValueError("native root ancestor is writable or unowned") + self._chain.append((parent, part, fd, (info.st_dev, info.st_ino))) + parent = fd + self.fd = parent + info = os.fstat(self.fd) + if info.st_uid != os.getuid() or (private and info.st_mode & 0o077): + raise ValueError("native root ownership or private mode differs") + except BaseException: + self.close() + raise + + def check(self): + for parent, name, fd, expected in self._chain: + current = os.stat(name, dir_fd=parent, follow_symlinks=False) + held = os.fstat(fd) + if (not stat.S_ISDIR(current.st_mode) + or (current.st_dev, current.st_ino) != expected + or (held.st_dev, held.st_ino) != expected + or current.st_uid not in (0, os.getuid()) + or current.st_mode & 0o022): + raise ValueError("native directory changed") + leaf = os.fstat(self.fd) + if leaf.st_uid != os.getuid() or (self._private and leaf.st_mode & 0o077): + raise ValueError("native private root permissions changed") + + def close(self): + self._stack.close() + + def __enter__(self): + return self + + def __exit__(self, *_): + self.close() + + def read(self, member: str, *, maximum: int = 512 * 1024 * 1024) -> bytes: + parts = relative_member(member) + self.check() + with ExitStack() as stack: + parent = self.fd + chain = [] + for part in parts[:-1]: + fd = os.open(part, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW, dir_fd=parent) + stack.callback(os.close, fd) + info = os.fstat(fd) + if info.st_uid != os.getuid() or info.st_mode & 0o022: + raise ValueError("native member ancestor is writable or unowned") + chain.append((parent, part, fd, (info.st_dev, info.st_ino))) + parent = fd + fd = os.open(parts[-1], os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK, dir_fd=parent) + stack.callback(os.close, fd) + before = os.fstat(fd) + if (not stat.S_ISREG(before.st_mode) or before.st_uid != os.getuid() + or before.st_mode & 0o022 or before.st_size > maximum): + raise ValueError("native member is not an owned bounded regular file") + chunks = [] + size = 0 + while size <= maximum: + chunk = os.read(fd, min(1024 * 1024, maximum + 1 - size)) + if not chunk: + break + size += len(chunk) + chunks.append(chunk) + current = os.stat(parts[-1], dir_fd=parent, follow_symlinks=False) + if (file_identity(before) != file_identity(os.fstat(fd)) + or file_identity(before) != file_identity(current) + or size != before.st_size or size > maximum): + raise ValueError("native member changed during read") + for ancestor, name, held, identity in chain: + actual = os.stat(name, dir_fd=ancestor, follow_symlinks=False) + if (not stat.S_ISDIR(actual.st_mode) + or (actual.st_dev, actual.st_ino) != identity + or (os.fstat(held).st_dev, os.fstat(held).st_ino) != identity): + raise ValueError("native member directory changed") + self.check() + return b"".join(chunks) + + def write_exclusive(self, member: str, raw: bytes): + """Publish a small direct child without overwriting an existing artifact.""" + if len(relative_member(member)) != 1 or type(raw) is not bytes or len(raw) > 1024 * 1024: + raise ValueError("native publication must be bounded direct-child bytes") + self.check() + fd = os.open(member, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, + 0o600, dir_fd=self.fd) + with os.fdopen(fd, "wb") as stream: + stream.write(raw) + stream.flush() + os.fsync(stream.fileno()) + self.check() + if self.read(member, maximum=1024 * 1024) != raw: + raise ValueError("native publication readback differs") + + def members(self, *, aliases: dict[str, str] | None = None) -> tuple[str, ...]: + """List every regular file, including importable bytecode and startup files.""" + self.check() + result = [] + aliases = aliases or {} + seen_aliases = set() + + def walk(fd, prefix, depth): + if depth > 32: + raise ValueError("native inventory depth exceeded") + for name in sorted(os.listdir(fd)): + member = f"{prefix}/{name}" if prefix else name + relative_member(member) + info = os.stat(name, dir_fd=fd, follow_symlinks=False) + if member in aliases: + if (not stat.S_ISLNK(info.st_mode) or info.st_uid != os.getuid() + or os.readlink(name, dir_fd=fd) != aliases[member]): + raise ValueError("native declared alias differs") + seen_aliases.add(member) + elif stat.S_ISDIR(info.st_mode): + child = os.open(name, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW, dir_fd=fd) + try: + if (info.st_dev, info.st_ino) != (os.fstat(child).st_dev, os.fstat(child).st_ino): + raise ValueError("native inventory directory changed") + if info.st_uid != os.getuid() or info.st_mode & 0o022: + raise ValueError("native inventory directory is writable or unowned") + walk(child, member, depth + 1) + finally: + os.close(child) + elif stat.S_ISREG(info.st_mode): + result.append(member) + if len(result) > 100000: + raise ValueError("native inventory member count exceeded") + else: + raise ValueError("native inventory contains a nonregular member") + walk(self.fd, "", 0) + if seen_aliases != set(aliases): + raise ValueError("native declared alias is absent") + self.check() + return tuple(sorted(result)) + + def inventory(self, *, maximum_total: int = 4 * 1024**3) -> tuple[NativeFile, ...]: + names = self.members() + result = [] + total = 0 + for name in names: + raw = self.read(name) + total += len(raw) + if total > maximum_total: + raise ValueError("native inventory byte budget exceeded") + result.append(NativeFile(name, len(raw), hashlib.sha256(raw).hexdigest())) + if self.members() != names: + raise ValueError("native inventory membership changed") + return tuple(result) diff --git a/tuner/runtime/native_macho.py b/tuner/runtime/native_macho.py new file mode 100644 index 00000000..22b47153 --- /dev/null +++ b/tuner/runtime/native_macho.py @@ -0,0 +1,136 @@ +"""Read arm64 Mach-O dependency commands without loading native code. + +The closure checker deliberately rejects unsupported runpath inheritance and +ambiguous resolutions. OS shared-cache paths are an explicit reviewed platform +trust boundary, not falsely hashed filesystem files. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path +import struct + + +@dataclass(frozen=True) +class MachOLinks: + dependencies: tuple[str, ...] + rpaths: tuple[str, ...] + + +def macho_links(raw: bytes) -> MachOLinks | None: + if raw[:4] in (b"\xca\xfe\xba\xbe", b"\xca\xfe\xba\xbf"): + wide = raw[:4] == b"\xca\xfe\xba\xbf" + if len(raw) < 8: + raise ValueError("truncated native fat header") + count = struct.unpack_from(">I", raw, 4)[0] + width = 32 if wide else 20 + if not 1 <= count <= 32 or 8 + count * width > len(raw): + raise ValueError("invalid native fat inventory") + slices = [] + for index in range(count): + offset = 8 + width * index + cpu = struct.unpack_from(">I", raw, offset)[0] + start, size = struct.unpack_from(">QQ" if wide else ">II", raw, offset + 8) + if start < 8 + count * width or start + size > len(raw): + raise ValueError("invalid native fat slice") + if cpu == 0x0100000C: + slices.append(raw[start:start + size]) + if len(slices) != 1: + raise ValueError("native fat file requires one arm64 slice") + return _thin_links(slices[0]) + if raw[:4] == b"\xcf\xfa\xed\xfe": + return _thin_links(raw) + if raw[:4] in (b"\xfe\xed\xfa\xcf", b"\xce\xfa\xed\xfe", b"\xfe\xed\xfa\xce", + b"\xbe\xba\xfe\xca", b"\xbf\xba\xfe\xca"): + raise ValueError("unsupported native Mach-O architecture") + return None + + +def _thin_links(raw): + if len(raw) < 32 or raw[:4] != b"\xcf\xfa\xed\xfe": + raise ValueError("invalid native Mach-O header") + cpu, _, _, count, size = struct.unpack_from(" 4096 or size > 1024 * 1024 or 32 + size > len(raw): + raise ValueError("invalid native Mach-O load commands") + dependencies, rpaths = [], [] + position = 32 + for _ in range(count): + if position + 8 > 32 + size: + raise ValueError("truncated native load command") + command, length = struct.unpack_from(" 32 + size: + raise ValueError("invalid native load command length") + # LC_LOAD_DYLIB, WEAK, REEXPORT, LAZY, UPWARD and LC_RPATH. + if command in (0xC, 0x80000018, 0x8000001F, 0x20, 0x80000023, 0x8000001C): + minimum = 12 if command == 0x8000001C else 24 + if length < minimum: + raise ValueError("truncated native dependency command") + name_offset = struct.unpack_from(" tuple[tuple[str, str, str], ...]: + """Resolve every observed native dependency into captured bytes or OS trust.""" + if (len(system_libraries) != len(set(system_libraries)) + or any(not (p.startswith("/usr/lib/") or p.startswith("/System/Library/")) + for p in system_libraries)): + raise ValueError("invalid native OS library trust set") + edges = [] + used_system = set() + + def expand(value, loader): + if value == "@loader_path" or value.startswith("@loader_path/"): + return str(Path(loader).parent) + value[len("@loader_path"):] + if value == "@executable_path" or value.startswith("@executable_path/"): + return str(Path(executable).parent) + value[len("@executable_path"):] + if not value.startswith("/"): + raise ValueError("unsupported native dependency resolution") + return value + + for loader, links in sorted(inventories.items()): + for dependency in links.dependencies: + if dependency in system_libraries: + used_system.add(dependency) + edges.append((loader, dependency, dependency)) + continue + candidates = ( + [expand(rpath, loader) + dependency[len("@rpath"):] for rpath in links.rpaths] + if dependency.startswith("@rpath/") + else [expand(dependency, loader)] + ) + resolved = set() + for candidate in candidates: + path = Path(candidate) + # Explicitly resolve Homebrew's stable aliases to reviewed bytes. + # The capture retains both the load spelling and canonical target; + # revalidation repeats resolution, detecting alias changes. + if path.exists(): + canonical = str(path.resolve(strict=True)) + if canonical not in admitted_files: + raise ValueError("native dependency is outside captured files") + resolved.add(canonical) + if len(resolved) != 1: + raise ValueError("native dependency is unresolved or ambiguous") + target = resolved.pop() + if target not in inventories: + raise ValueError("native dependency target is not arm64 Mach-O") + edges.append((loader, dependency, target)) + if used_system != set(system_libraries): + raise ValueError("native OS library trust set contains unused members") + return tuple(edges) diff --git a/tuner/runtime/native_mlx_artifacts.py b/tuner/runtime/native_mlx_artifacts.py new file mode 100644 index 00000000..b0554b04 --- /dev/null +++ b/tuner/runtime/native_mlx_artifacts.py @@ -0,0 +1,113 @@ +"""Independent bounded MLX safetensors verification, with no ML imports. + +An MLX adapter is not labeled PEFT. Final validation belongs to the explicitly +reloaded saved adapter; upstream in-loop validation is never final evidence. +""" + +from __future__ import annotations + +from dataclasses import dataclass +import hashlib +import math +import struct + +from tuner.runtime.native_mlx_material import strict_json + + +_WIDTH = {"F16": 2, "BF16": 2, "F32": 4} + + +@dataclass(frozen=True) +class NativeTensor: + name: str + dtype: str + shape: tuple[int, ...] + start: int + end: int + + +def adapter_tensors(raw: bytes) -> tuple[NativeTensor, ...]: + if type(raw) is not bytes or not 10 <= len(raw) <= 512 * 1024 * 1024: + raise ValueError("native adapter byte budget exceeded") + header_size = struct.unpack_from(" len(raw): + raise ValueError("native tensor bounds differ") + tensors.append(NativeTensor(name, dtype, tuple(shape), payload_start + start, payload_start + end)) + if not tensors: + raise ValueError("native adapter has no tensors") + cursor = payload_start + for tensor in sorted(tensors, key=lambda t: t.start): + if tensor.start != cursor: + raise ValueError("native adapter has overlap or unclaimed bytes") + cursor = tensor.end + if cursor != len(raw): + raise ValueError("native adapter has trailing bytes") + return tuple(sorted(tensors, key=lambda t: t.name)) + + +def _values(raw, tensor): + view = memoryview(raw)[tensor.start:tensor.end] + if tensor.dtype == "BF16": + for (bits,) in struct.iter_unpack(" tuple[NativeTensor, ...]: + tensors = adapter_tensors(raw) + if any(not math.isfinite(value) for tensor in tensors for value in _values(raw, tensor)): + raise ValueError("native adapter has nonfinite parameters") + return tensors + + +def verify_adapter_delta(initial: bytes, final: bytes, *, expected_shapes: tuple[tuple[str, tuple[int, ...]], ...]): + before = adapter_tensors(initial) + after = adapter_tensors(final) + if (tuple((t.name, t.shape) for t in before) != expected_shapes + or tuple((t.name, t.shape) for t in after) != expected_shapes + or tuple(t.dtype for t in before) != tuple(t.dtype for t in after)): + raise ValueError("native adapter parameter inventory differs") + count, changed, norm = 0, 0, 0.0 + for old_tensor, new_tensor in zip(before, after): + for old, new in zip(_values(initial, old_tensor), _values(final, new_tensor)): + if not math.isfinite(old) or not math.isfinite(new): + raise ValueError("native adapter has nonfinite parameters") + delta = new - old + count += 1 + changed += delta != 0 + norm += delta * delta + if changed == 0 or not math.isfinite(norm) or norm <= 0: + raise ValueError("native adapter has no finite parameter update") + return { + "initial_sha256": hashlib.sha256(initial).hexdigest(), + "final_sha256": hashlib.sha256(final).hexdigest(), + "parameter_count": count, "changed_parameters": changed, + "delta_l2": math.sqrt(norm), + } diff --git a/tuner/runtime/native_mlx_child.py b/tuner/runtime/native_mlx_child.py new file mode 100644 index 00000000..27c872ad --- /dev/null +++ b/tuner/runtime/native_mlx_child.py @@ -0,0 +1,228 @@ +"""Isolated native worker entrypoint for the admitted owned-process transport. + +Invoke with -I -S -B and an absolute private request path. This entrypoint grants +no authority itself: the host/Foundation must bind the exact request and runtime +before dispatch. No user-facing native launch command is registered. +""" + +from __future__ import annotations + +import hashlib +import importlib.machinery +import io +import json +import os +from pathlib import Path +import stat +import sys + + +def _read(path, maximum): + path = Path(path) + if not path.is_absolute(): + raise ValueError("native bootstrap requires absolute paths") + fd = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK) + try: + before = os.fstat(fd) + if (not stat.S_ISREG(before.st_mode) or before.st_uid != os.getuid() + or before.st_mode & 0o022 or not 0 <= before.st_size <= maximum): + raise ValueError("native bootstrap file is invalid") + chunks, size = [], 0 + while size <= maximum: + chunk = os.read(fd, min(1024 * 1024, maximum + 1 - size)) + if not chunk: + break + chunks.append(chunk) + size += len(chunk) + after, current = os.fstat(fd), path.lstat() + identity = lambda s: (s.st_dev, s.st_ino, s.st_size, s.st_mtime_ns, s.st_ctime_ns) + if identity(before) != identity(after) or identity(before) != identity(current) or size != before.st_size: + raise ValueError("native bootstrap file changed") + return b"".join(chunks) + finally: + os.close(fd) + + +def _json(raw): + def pairs(items): + result = {} + for key, value in items: + if key in result: + raise ValueError("duplicate native bootstrap field") + result[key] = value + return result + + def reject(_): + raise ValueError("nonfinite native bootstrap number") + return json.loads(raw, object_pairs_hook=pairs, parse_constant=reject) + + +def _canonical(value): + return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, + allow_nan=False).encode() + + +class _PinnedSourceLoader(importlib.machinery.SourceFileLoader): + def __init__(self, fullname, filename, digest): + super().__init__(fullname, filename) + self.digest = digest + + def get_code(self, fullname): + raw = _read(self.path, 64 * 1024 * 1024) + if hashlib.sha256(raw).hexdigest() != self.digest: + raise ImportError("native source bytes differ") + # Compile pinned source; never execute an ambient timestamp-based cache. + return self.source_to_code(raw, self.path) + + +class _PinnedFinder: + def __init__(self, members): + self.members = members + + def find_spec(self, fullname, path=None, target=None): + spec = importlib.machinery.PathFinder.find_spec(fullname, path) + if spec is None: + return None + if spec.origin is None: + if not spec.submodule_search_locations or any( + not any(Path(location) in Path(member).parents for member in self.members) + for location in spec.submodule_search_locations + ): + raise ImportError("native namespace origin is outside capture") + return spec + expected = self.members.get(str(Path(spec.origin).resolve(strict=True))) + if expected is None: + raise ImportError("native module origin is outside capture") + raw = _read(spec.origin, 512 * 1024 * 1024) + if hashlib.sha256(raw).hexdigest() != expected: + raise ImportError("native imported module bytes differ") + if isinstance(spec.loader, importlib.machinery.SourceFileLoader): + spec.loader = _PinnedSourceLoader(fullname, spec.origin, expected) + elif not isinstance(spec.loader, importlib.machinery.ExtensionFileLoader): + raise ImportError("native loader type is unsupported") + return spec + + +def install_capture_imports(capture): + config = capture["config"] + members = {} + for tree in capture["trees"]: + for item in tree["members"]: + path = str(Path(tree["root"]) / item["path"]) + if path in members: + raise ValueError("native import inventory contains duplicates") + members[path] = item["sha256"] + for item in capture["files"]: + if item["path"] in members: + raise ValueError("native import inventory contains duplicates") + members[item["path"]] = item["sha256"] + # No user site, global site-packages, current directory or zip search path. + source_parents = tuple(dict.fromkeys(str(Path(root).parent) for root in config["sources"])) + sys.path[:] = [config["stdlib"], str(Path(config["stdlib"]) / "lib-dynload"), + *source_parents, config["packages"]] + for name, module in tuple(sys.modules.items()): + origin = getattr(module, "__file__", None) + if origin is None: + continue + canonical_origin = str(Path(origin).resolve(strict=True)) + if canonical_origin not in members: + raise ImportError("native bootstrap loaded an uncaptured module") + if hashlib.sha256(_read(origin, 512 * 1024 * 1024)).hexdigest() != members[canonical_origin]: + raise ImportError("native bootstrap module bytes differ") + finder = _PinnedFinder(members) + sys.meta_path.insert(0, finder) + return finder + + +def _snapshot(value): + from tuner.runtime.native_files import NativeFile + from tuner.runtime.native_mlx_input import NativeModelSnapshot + if type(value) is not dict or set(value) != {"repository", "revision", "root", "files"}: + raise ValueError("native child model snapshot fields differ") + if type(value["files"]) is not list: + raise ValueError("native child snapshot inventory is invalid") + return NativeModelSnapshot( + value["repository"], value["revision"], value["root"], + tuple(NativeFile(**item) for item in value["files"]), + ) + + +def run_request(request, *, request_root): + """After isolated import admission; all workload controls are recompiled.""" + from tuner.runtime.native_files import NativeTree + from tuner.runtime.native_mlx_composition import execute_native_sft + from tuner.runtime.native_mlx_input import prepare_native_input + from tuner.training.contracts import RetainedTrainingInputStreamLease + from tuner.training.native_mlx_compilation import parse_native_workload + + fields = {"schema", "workload", "runtime_capture", "dependency_lock", "dataset", + "model_snapshot", "tokenizer_snapshot", "output"} + if type(request) is not dict or set(request) != fields or request["schema"] != "synaptic-native-child-request/v1": + raise ValueError("native child request fields differ") + projection = parse_native_workload(_canonical(request["workload"])) + profile = projection.profile + model, tokenizer = _snapshot(request["model_snapshot"]), _snapshot(request["tokenizer_snapshot"]) + if (model.repository, model.revision, tokenizer.repository, tokenizer.revision) != ( + profile.model_ref, profile.model_revision, profile.model_ref, profile.tokenizer_revision, + ): + raise ValueError("native snapshots differ from compiled model/tokenizer pins") + for key in ("runtime_capture", "dependency_lock", "dataset", "output"): + if type(request[key]) is not str or Path(request[key]).parent != request_root: + raise ValueError("native child transport is outside its private request root") + with NativeTree(request_root, private=True) as tree: + runtime = tree.read(Path(request["runtime_capture"]).name, maximum=16 * 1024 * 1024) + lock = tree.read(Path(request["dependency_lock"]).name, maximum=1024 * 1024) + dataset = tree.read(Path(request["dataset"]).name, maximum=64 * 1024 * 1024) + + class Source: + identity = projection.source + + def open_lease(self): + return RetainedTrainingInputStreamLease(self.identity, io.BytesIO(dataset)) + bound = prepare_native_input( + source=Source(), identity=projection.source, tokenizer_snapshot=tokenizer, + tokenization=projection.tokenization, config=projection.controls, + workload_digest=projection.workload.fingerprint, + runtime_material=runtime, runtime_digest=profile.runtime_digest, dependency_lock=lock, + ) + result = execute_native_sft( + bound=bound, model_snapshot=model, lora=projection.lora, optimizer=projection.optimizer, + output=Path(request["output"]), runtime_material=runtime, + runtime_digest=profile.runtime_digest, dependency_lock=lock, + ) + with NativeTree(Path(request["output"]), private=True) as output: + output.write_exclusive("workload.json", projection.workload.canonical_bytes) + tree.check() + return result + + +def main(): + # Diagnostics disclose phases only, never exception/source/provider contents. + phase = 120 + try: + if not sys.flags.isolated or not sys.flags.no_site or not sys.dont_write_bytecode: + raise ValueError("native child must run with -I -S -B") + if len(sys.argv) != 3 or sys.argv[1] != "--request": + raise ValueError("native child arguments differ") + path = Path(sys.argv[2]) + request = _json(_read(path, 2 * 1024 * 1024)) + runtime_path = Path(request["runtime_capture"]) + if runtime_path.parent != path.parent: + raise ValueError("native capture is outside the request root") + runtime = _read(runtime_path, 16 * 1024 * 1024) + if hashlib.sha256(runtime).hexdigest() != request["workload"]["profile"]["runtime_digest"]: + raise ValueError("native capture digest differs from workload") + capture = _json(runtime) + if capture["schema"] != "synaptic-native-mlx-capture/v1" or _canonical(capture) != runtime: + raise ValueError("native capture schema differs") + phase = 121 + install_capture_imports(capture) + phase = 122 + run_request(request, request_root=path.parent) + return 0 + except BaseException: + return phase + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tuner/runtime/native_mlx_composition.py b/tuner/runtime/native_mlx_composition.py new file mode 100644 index 00000000..f27f5938 --- /dev/null +++ b/tuner/runtime/native_mlx_composition.py @@ -0,0 +1,217 @@ +"""Explicit candidate MLX model/LoRA/optimizer composition in an admitted child. + +This is not a launcher or public provider. The bounded supervisor and the host +grant must precede entry. No Hub download or implicit model/profile selection. +""" + +from __future__ import annotations + +from dataclasses import asdict, dataclass +import hashlib +import math +from pathlib import Path +import re + +from tuner.runtime.native_files import NativeTree +from tuner.runtime.native_mlx_artifacts import adapter_tensors, verify_adapter_delta +from tuner.runtime.native_mlx_input import BoundNativeInput, NativeModelSnapshot +from tuner.runtime.native_mlx_material import canonical, strict_json, verify_native_material +from tuner.runtime.native_mlx_runtime import MLXRuntimeExpectation +from tuner.runtime.native_mlx_sft import make_mlx_sft_hooks, run_prepared_mlx_sft + + +def _number(value, minimum, maximum, *, closed=False): + if type(value) not in (int, float) or not math.isfinite(value): + raise ValueError("native optimizer/LoRA number is invalid") + if not (minimum <= value <= maximum if closed else minimum < value <= maximum): + raise ValueError("native optimizer/LoRA number is outside bounds") + + +@dataclass(frozen=True) +class NativeLoRA: + layers: int + rank: int + scale: float + dropout: float + targets: tuple[str, ...] + + def __post_init__(self): + if (type(self.layers) is not int or not 1 <= self.layers <= 4096 + or type(self.rank) is not int or not 1 <= self.rank <= 4096): + raise ValueError("native LoRA rank/layer count is invalid") + _number(self.scale, 0, 65536) + _number(self.dropout, 0, 1, closed=True) + if self.dropout == 1: + raise ValueError("native LoRA dropout must be below one") + if (type(self.targets) is not tuple or not 1 <= len(self.targets) <= 256 + or self.targets != tuple(sorted(set(self.targets))) + or any(type(key) is not str or not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_.]*", key) + for key in self.targets)): + raise ValueError("native LoRA targets must be unique explicit layer-relative keys") + + +@dataclass(frozen=True) +class NativeAdamW: + learning_rate: float + beta1: float + beta2: float + epsilon: float + weight_decay: float + bias_correction: bool + schedule: str + + def __post_init__(self): + _number(self.learning_rate, 0, 1) + _number(self.epsilon, 0, 1) + for value in (self.beta1, self.beta2, self.weight_decay): + _number(value, 0, 1, closed=True) + if self.beta1 == 1 or self.beta2 == 1: + raise ValueError("native AdamW beta must be below one") + if type(self.bias_correction) is not bool or self.schedule != "constant": + raise ValueError("native AdamW requires explicit constant schedule and bias policy") + + def arguments(self): + return { + "learning_rate": self.learning_rate, "betas": [self.beta1, self.beta2], + "eps": self.epsilon, "weight_decay": self.weight_decay, + "bias_correction": self.bias_correction, + } + + +def attach_native_lora(model, config: NativeLoRA, *, convert, flatten): + """Verify every selected target exists and only those adapters are trainable.""" + if type(config) is not NativeLoRA: + raise TypeError("native LoRA controls must be exact") + layers = model.layers + if not config.layers <= len(layers): + raise ValueError("native LoRA layer selection exceeds model") + global_names = {} + for name, module in model.named_modules(): + global_names.setdefault(id(module), []).append(name) + expected = [] + for layer in layers[-config.layers:]: + modules = dict(layer.named_modules()) + for key in config.targets: + if key not in modules or len(global_names.get(id(modules[key]), [])) != 1: + raise ValueError("native LoRA target is missing or aliased") + full_name = global_names[id(modules[key])][0] + expected.extend((full_name + ".lora_a", full_name + ".lora_b")) + if len(set(expected)) != len(expected): + raise ValueError("native LoRA target is shared between layers") + model.freeze() + convert(model, config.layers, { + "rank": config.rank, "scale": config.scale, "dropout": config.dropout, + "keys": list(config.targets), + }, use_dora=False) + trainable = dict(flatten(model.trainable_parameters())) + if tuple(sorted(trainable)) != tuple(sorted(expected)): + raise ValueError("native realized trainable parameters differ from configured targets") + shapes = tuple((name, tuple(value.shape)) for name, value in sorted(trainable.items())) + if any(len(shape) != 2 or config.rank not in shape for _, shape in shapes): + raise ValueError("native realized LoRA dimensions differ") + return trainable, shapes + + +def execute_native_sft( + *, bound: BoundNativeInput, model_snapshot: NativeModelSnapshot, + lora: NativeLoRA, optimizer: NativeAdamW, output: Path, + runtime_material: bytes, runtime_digest: str, dependency_lock: bytes, +) -> bytes: + """Produce final saved-adapter evidence only after exact offline readback. + + A successful return is still candidate worker evidence, not an authenticated + public artifact manifest or hardware capability promotion. + """ + if (type(bound) is not BoundNativeInput or type(model_snapshot) is not NativeModelSnapshot + or type(lora) is not NativeLoRA or type(optimizer) is not NativeAdamW + or bound.runtime_digest != runtime_digest): + raise ValueError("native composition binding differs") + verify_native_material(runtime_material, expected_digest=runtime_digest, dependency_lock=dependency_lock) + runtime = strict_json(runtime_material)["runtime"] + expectation = MLXRuntimeExpectation.from_dict({ + name: runtime[name] for name in MLXRuntimeExpectation.__dataclass_fields__ + }) + with NativeTree(Path(model_snapshot.root), private=True) as source, NativeTree(output, private=True) as destination: + model_snapshot.verify(source) + if destination.members(): + raise ValueError("native training output must be fresh and empty") + import mlx.core as mx + import mlx.nn as nn + import mlx.optimizers as optimizers + from mlx.utils import tree_flatten + from mlx_lm.utils import load_model + from mlx_lm.tuner.utils import linear_to_lora_layers + from mlx_lm.tuner.trainer import evaluate + + mx.random.seed(bound.prepared.config.seed) + # load_model accepts a Path and does not pass through Hub _download. + model, _ = load_model(Path(model_snapshot.root), lazy=False, strict=True, + trust_remote_code=False) + parameters, shapes = attach_native_lora( + model, lora, convert=linear_to_lora_layers, flatten=tree_flatten, + ) + mx.eval(parameters) + initial_path = output / "initial-adapters.safetensors" + final_path = output / "adapters.safetensors" + mx.save_safetensors(str(initial_path), parameters) + initial = destination.read(initial_path.name) + adapter_tensors(initial) + adam = optimizers.AdamW(**optimizer.arguments()) + run_prepared_mlx_sft( + bound.prepared, model=model, optimizer=adam, adapter_file=str(final_path), + runtime=expectation, dependency_lock=dependency_lock, + ) + mx.eval(adam.state) + completed_updates = adam.step.item() + if type(completed_updates) is not int or completed_updates != bound.prepared.config.iterations: + raise ValueError("native optimizer update count differs") + final = destination.read(final_path.name) + delta = verify_adapter_delta(initial, final, expected_shapes=shapes) + saved = mx.load(str(final_path)) + if tuple(sorted(saved)) != tuple(name for name, _ in shapes): + raise ValueError("native saved adapter inventory differs") + model.load_weights(list(saved.items()), strict=False) + mx.eval(model.parameters()) + # Upstream evaluates before its final update. Evaluate the saved adapter + # separately after loading it, including the partial validation batch. + iterate, loss = make_mlx_sft_hooks(bound.prepared, mx=mx, nn=nn) + final_loss = evaluate( + model=model, dataset=bound.prepared.validation, + batch_size=bound.prepared.config.batch_size, num_batches=-1, + max_seq_length=bound.prepared.config.max_seq_length, + loss=loss, iterate_batches=iterate, clear_cache_threshold=0, + ) + if type(final_loss) not in (int, float) or not math.isfinite(final_loss) or final_loss < 0: + raise ValueError("native saved final adapter validation is nonfinite") + if destination.read(final_path.name) != final: + raise ValueError("native final adapter changed during evaluation") + model_snapshot.verify(source) + destination.check() + adapter_config = canonical({ + "fine_tune_type": "lora", "num_layers": lora.layers, + "lora_parameters": { + "rank": lora.rank, "scale": lora.scale, "dropout": lora.dropout, + "keys": list(lora.targets), + }, + }) + destination.write_exclusive("adapter_config.json", adapter_config) + destination.write_exclusive("input_binding.json", canonical(bound.to_dict())) + result = canonical({ + "schema": "synaptic-native-mlx-sft-result/v1", + "status": "CANDIDATE_WORKER_COMPLETE", + "input_binding_digest": bound.digest, "workload_digest": bound.workload_digest, + "runtime_digest": runtime_digest, "model_snapshot_digest": model_snapshot.digest, + "lora": asdict(lora), "optimizer": asdict(optimizer), + "completed_updates": completed_updates, + "adapter_format": "mlx-lm-lora-safetensors", + "parameter_delta": delta, + "final_validation": { + "adapter_sha256": hashlib.sha256(final).hexdigest(), + "phase": "reloaded_saved_final_adapter", + "loss": final_loss, "rows": len(bound.prepared.validation), + "supervised_tokens": sum(sum(label != -100 for label in row.labels[1:]) + for row in bound.prepared.validation), + }, + }) + destination.write_exclusive("native_result.json", result) + return result diff --git a/tuner/runtime/native_mlx_input.py b/tuner/runtime/native_mlx_input.py new file mode 100644 index 00000000..b57f2f2d --- /dev/null +++ b/tuner/runtime/native_mlx_input.py @@ -0,0 +1,194 @@ +"""Native child join from retained prepared publication to shared tokenization. + +No dataset normalization, re-split, mask repair or network access. Publication +lineage/format validation remains the existing host preparation service's job. +This join verifies that exact retained bytes are what the tokenizer consumes. +""" + +from __future__ import annotations + +from dataclasses import asdict, dataclass +import hashlib +from pathlib import Path +import re + +from shared.sft_preprocessing import materialize_sft_example +from synaptic_tuner.api.v1.training_sources import PreparedTrainingInputIdentity +from tuner.runtime.native_files import NativeFile, NativeTree +from tuner.runtime.native_mlx_material import canonical, strict_json, verify_native_material +from tuner.runtime.native_mlx_sft import MLXSFTConfig, PreparedMLXSFT, prepare_mlx_sft +from tuner.training.contracts import RetainedTrainingInputStreamLease + + +@dataclass(frozen=True) +class NativeModelSnapshot: + """Reviewed host material, not self-issued download or model provenance.""" + + repository: str + revision: str + root: str + files: tuple[NativeFile, ...] + + def __post_init__(self): + if (type(self.repository) is not str or not re.fullmatch( + r"[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+", self.repository) + or type(self.revision) is not str or not re.fullmatch(r"[a-f0-9]{40}", self.revision)): + raise ValueError("native model source pins are invalid") + if type(self.root) is not str or not Path(self.root).is_absolute(): + raise ValueError("native model root must be absolute") + if (type(self.files) is not tuple or not self.files + or any(type(member) is not NativeFile for member in self.files) + or tuple(f.path for f in self.files) != tuple(sorted({f.path for f in self.files}))): + raise ValueError("native model inventory is invalid") + + @property + def digest(self): + return hashlib.sha256(canonical(asdict(self))).hexdigest() + + def verify(self, tree: NativeTree): + if str(tree.path) != self.root or tree.inventory() != self.files: + raise ValueError("native model snapshot bytes differ") + + +@dataclass(frozen=True) +class NativeTokenization: + assistant_only_loss: bool + prompt_render: str + chat_template_json: bytes | None + use_fast: bool + + def __post_init__(self): + if type(self.assistant_only_loss) is not bool or type(self.use_fast) is not bool: + raise ValueError("native tokenizer switches must be explicit booleans") + if self.prompt_render not in ("full_conversation", "prompt_completion"): + raise ValueError("native prompt render is unsupported") + if self.chat_template_json is not None: + parsed = strict_json(self.chat_template_json, maximum=65536) + if type(parsed) is not dict or canonical(parsed) != self.chat_template_json: + raise ValueError("native template arguments must be canonical JSON") + + def to_dict(self): + return { + "assistant_only_loss": self.assistant_only_loss, + "prompt_render": self.prompt_render, + "chat_template_kwargs": ( + None if self.chat_template_json is None else strict_json(self.chat_template_json) + ), + "use_fast": self.use_fast, + } + + +@dataclass(frozen=True) +class BoundNativeInput: + prepared: PreparedMLXSFT + tokenizer_snapshot_digest: str + tokenization: NativeTokenization + workload_digest: str + runtime_digest: str + + def to_dict(self): + return { + "schema": "synaptic-native-tokenization-binding/v1", + "prepared_digest": self.prepared.digest, + "source": self.prepared.source.to_dict(), + "tokenizer_snapshot_digest": self.tokenizer_snapshot_digest, + "tokenization": self.tokenization.to_dict(), + "workload_digest": self.workload_digest, + "runtime_digest": self.runtime_digest, + } + + @property + def digest(self): + return hashlib.sha256(canonical(self.to_dict())).hexdigest() + + +def read_retained_native_source(source, identity: PreparedTrainingInputIdentity) -> bytes: + if (type(identity) is not PreparedTrainingInputIdentity or source.identity != identity + or not 1 <= identity.size_bytes <= 64 * 1024 * 1024): + raise ValueError("native prepared source identity differs") + lease = source.open_lease() + if type(lease) is not RetainedTrainingInputStreamLease: + raise TypeError("native source must return the existing retained stream lease") + try: + if lease.identity != identity: + raise ValueError("native prepared lease identity differs") + stream = lease.take_stream() + chunks, size = [], 0 + while size <= identity.size_bytes: + chunk = stream.read(min(1024 * 1024, identity.size_bytes + 1 - size)) + if type(chunk) is not bytes: + raise TypeError("native source must stream bytes") + if not chunk: + break + size += len(chunk) + chunks.append(chunk) + raw = b"".join(chunks) + if len(raw) != identity.size_bytes or hashlib.sha256(raw).hexdigest() != identity.content_digest: + raise ValueError("native prepared source bytes differ") + return raw + finally: + lease.close() + + +def tokenize_native_source( + *, source, identity: PreparedTrainingInputIdentity, tokenizer, + tokenizer_snapshot: NativeModelSnapshot, tokenization: NativeTokenization, + config: MLXSFTConfig, workload_digest: str, runtime_digest: str, +) -> BoundNativeInput: + """Internal algebra boundary. The child wrapper owns tokenizer admission.""" + for digest in (workload_digest, runtime_digest): + if type(digest) is not str or not re.fullmatch(r"[a-f0-9]{64}", digest): + raise ValueError("native input workload/runtime binding is invalid") + if type(tokenizer_snapshot) is not NativeModelSnapshot or type(tokenization) is not NativeTokenization: + raise TypeError("native input requires exact tokenizer controls") + raw = read_retained_native_source(source, identity) + splits = {"train": [], "validation": []} + lines = raw.splitlines() + if not 2 <= len(lines) <= 100000: + raise ValueError("native prepared row count is invalid") + for line in lines: + record = strict_json(line, maximum=1024 * 1024) + if type(record) is not dict or record.get("schema_version") != identity.format: + raise ValueError("native prepared format differs from publication") + # These are the established publication fields, not an MLX text schema. + if record.get("split") not in splits: + raise ValueError("native prepared row has an unsupported declared split") + row = materialize_sft_example( + tokenizer=tokenizer, record=record, max_seq_length=config.max_seq_length, + assistant_only_loss=tokenization.assistant_only_loss, + source_hash=hashlib.sha256(line).hexdigest(), + chat_template_kwargs=tokenization.to_dict()["chat_template_kwargs"], + prompt_render=tokenization.prompt_render, + ) + splits[record["split"]].append((record.get("row_id"), row)) + prepared = prepare_mlx_sft( + source=identity, tokenizer_revision=tokenizer_snapshot.revision, + train=tuple(splits["train"]), validation=tuple(splits["validation"]), config=config, + ) + return BoundNativeInput(prepared, tokenizer_snapshot.digest, tokenization, + workload_digest, runtime_digest) + + +def prepare_native_input( + *, source, identity, tokenizer_snapshot, tokenization, config, + workload_digest: str, runtime_material: bytes, runtime_digest: str, dependency_lock: bytes, +) -> BoundNativeInput: + """Run only in the isolated admitted child, before model/Metal imports.""" + verify_native_material(runtime_material, expected_digest=runtime_digest, + dependency_lock=dependency_lock) + with NativeTree(Path(tokenizer_snapshot.root), private=True) as tree: + tokenizer_snapshot.verify(tree) + from transformers import AutoTokenizer + + tokenizer = AutoTokenizer.from_pretrained( + tokenizer_snapshot.root, local_files_only=True, trust_remote_code=False, + use_fast=tokenization.use_fast, + ) + result = tokenize_native_source( + source=source, identity=identity, tokenizer=tokenizer, + tokenizer_snapshot=tokenizer_snapshot, tokenization=tokenization, config=config, + workload_digest=workload_digest, runtime_digest=runtime_digest, + ) + # Hold the root identity across preparation, then recheck bytes. + tokenizer_snapshot.verify(tree) + return result diff --git a/tuner/runtime/native_mlx_material.py b/tuner/runtime/native_mlx_material.py new file mode 100644 index 00000000..24f05890 --- /dev/null +++ b/tuner/runtime/native_mlx_material.py @@ -0,0 +1,265 @@ +"""Reviewable native runtime bytes and linked-library closure. + +Capture never grants execution authority. The consuming host must independently +review and bind the exact capture digest in its grant and compiled workload. +OS shared-cache libraries are trusted through the exact observed OS build; +runtime-loaded plug-ins and hardware behavior still require qualification. +""" + +from __future__ import annotations + +from dataclasses import asdict, dataclass +import hashlib +import json +from pathlib import Path +import platform +import sys +import sysconfig + +from tuner.runtime.native_files import NativeTree +from tuner.runtime.native_macho import macho_links, resolve_links +from tuner.runtime.native_mlx_runtime import MLXRuntimeExpectation, inspect_mlx_runtime_candidate + + +def canonical(value) -> bytes: + return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, + allow_nan=False).encode("utf-8") + + +def strict_json(raw: bytes, *, maximum: int = 16 * 1024 * 1024): + if type(raw) is not bytes or not 1 <= len(raw) <= maximum: + raise ValueError("native document size is invalid") + + def pairs(items): + value = {} + for key, item in items: + if key in value: + raise ValueError("native document has duplicate keys") + value[key] = item + return value + + def constant(_): + raise ValueError("native document has nonfinite values") + + return json.loads(raw, object_pairs_hook=pairs, parse_constant=constant) + + +def _absolute(value): + if (type(value) is not str or not value.startswith("/") or len(value.encode()) > 4096 + or str(Path(value)) != value or any(p == ".." for p in Path(value).parts) + or any(ord(c) < 32 for c in value)): + raise ValueError("native capture path is invalid") + return value + + +@dataclass(frozen=True) +class NativeCaptureConfig: + """Named root trees plus explicit external interpreter/library files. + + ``stdlib`` and ``packages`` must be exact real directories; ``sources`` is + one or more narrow code trees, never a project root with secrets. Every file + in the selected trees participates, including .pth and .pyc files. + """ + + stdlib: str + packages: str + sources: tuple[str, ...] + files: tuple[str, ...] + system_libraries: tuple[str, ...] + aliases: tuple[tuple[str, str], ...] = () + + def __post_init__(self): + _absolute(self.stdlib) + _absolute(self.packages) + for name in ("sources", "files", "system_libraries"): + values = getattr(self, name) + if type(values) is not tuple or not 1 <= len(values) <= 4096: + raise ValueError("native capture path set is invalid") + if values != tuple(sorted(set(values))): + raise ValueError("native capture paths must be unique and sorted") + for value in values: + _absolute(value) + trees = (self.stdlib, self.packages, *self.sources) + if len(set(trees)) != len(trees): + raise ValueError("native capture trees overlap") + for first in trees: + for second in trees: + if first != second and Path(first).is_relative_to(second): + raise ValueError("native capture trees overlap") + if any(Path(file).is_relative_to(tree) for file in self.files for tree in trees): + raise ValueError("native explicit file duplicates a tree") + if (type(self.aliases) is not tuple or len(self.aliases) > 128 + or any(type(item) is not tuple or len(item) != 2 for item in self.aliases)): + raise ValueError("native aliases are invalid") + paths = [] + for path, target in self.aliases: + _absolute(path) + if type(target) is not str or not target or len(target.encode()) > 4096: + raise ValueError("native alias target is invalid") + if not any(Path(path).is_relative_to(tree) for tree in trees): + raise ValueError("native alias is outside capture") + paths.append(path) + if paths != sorted(set(paths)): + raise ValueError("native aliases must be unique and sorted") + + @classmethod + def from_dict(cls, value): + if type(value) is not dict or set(value) != set(cls.__dataclass_fields__): + raise ValueError("native capture configuration fields differ") + if any(type(value[name]) is not list for name in ("sources", "files", "system_libraries", "aliases")): + raise ValueError("native capture arrays are invalid") + return cls(value["stdlib"], value["packages"], tuple(value["sources"]), + tuple(value["files"]), tuple(value["system_libraries"]), + tuple(tuple(item) for item in value["aliases"])) + + +def capture_native_material(config: NativeCaptureConfig, expected: MLXRuntimeExpectation, + *, dependency_lock: bytes) -> bytes: + if type(config) is not NativeCaptureConfig or type(expected) is not MLXRuntimeExpectation: + raise TypeError("native capture requires exact configuration") + if str(Path(sysconfig.get_path("stdlib")).resolve(strict=True)) != config.stdlib: + raise ValueError("native standard library root differs") + executable = str(Path(sys.executable).resolve(strict=True)) + if executable not in config.files: + raise ValueError("native interpreter is absent from capture") + # Distribution discovery must use the same package tree whose bytes we bind. + from importlib import metadata + observed_roots = { + str(Path(item.locate_file("")).resolve(strict=True)) + for item in metadata.distributions() + } + if observed_roots != {config.packages}: + raise ValueError("native package discovery roots differ") + runtime = inspect_mlx_runtime_candidate(expected, dependency_lock=dependency_lock) + roots = (config.stdlib, config.packages, *config.sources) + alias_targets = [] + for path, target in config.aliases: + resolved = str(Path(path).resolve(strict=True)) + if path == str(Path(config.stdlib) / "site-packages"): + # Homebrew's shared site-packages link is explicitly excluded from + # the isolated -S child's search path, never followed or admitted. + if resolved == config.packages: + raise ValueError("native isolated package root aliases global site-packages") + disposition = "excluded-global-site-packages" + elif resolved in config.files: + disposition = "captured-file-alias" + else: + raise ValueError("native alias does not resolve to captured file bytes") + alias_targets.append({"path": path, "spelling": target, "resolved": resolved, + "disposition": disposition}) + inventories = [] + native = {} + all_files = set() + total = 0 + # Captured at one pass, then rechecked by the eventual child. Exclusive host + # ownership is required; this is not a filesystem-wide atomic snapshot. + for root in roots: + with NativeTree(Path(root)) as tree: + aliases = {str(Path(path).relative_to(root)): target for path, target in config.aliases + if Path(path).is_relative_to(root)} + names = tree.members(aliases=aliases) + members = [] + for member in names: + raw = tree.read(member) + total += len(raw) + if total > 4 * 1024**3: + raise ValueError("native capture aggregate budget exceeded") + absolute = str(Path(root) / member) + all_files.add(absolute) + members.append({"path": member, "size": len(raw), + "sha256": hashlib.sha256(raw).hexdigest()}) + links = macho_links(raw) + if Path(member).suffix in (".so", ".dylib") and links is None: + raise ValueError("native extension has unsupported executable format") + if links is not None: + native[absolute] = links + if tree.members(aliases=aliases) != names: + raise ValueError("native capture membership changed") + inventories.append({"root": root, "members": members}) + files = [] + for filename in config.files: + path = Path(filename) + with NativeTree(path.parent) as tree: + raw = tree.read(path.name) + total += len(raw) + if total > 4 * 1024**3: + raise ValueError("native capture aggregate budget exceeded") + files.append({"path": filename, "size": len(raw), + "sha256": hashlib.sha256(raw).hexdigest()}) + all_files.add(filename) + links = macho_links(raw) + if path.suffix in (".so", ".dylib") and links is None: + raise ValueError("native external library has unsupported executable format") + if links is not None: + native[filename] = links + if executable not in native: + raise ValueError("native interpreter is not arm64 Mach-O") + graph = resolve_links(native, executable=executable, admitted_files=all_files, + system_libraries=config.system_libraries) + return canonical({ + "schema": "synaptic-native-mlx-capture/v1", + "status": "OBSERVED_NOT_ADMITTED", + "platform": {"system": platform.system(), "machine": platform.machine(), + "macos": platform.mac_ver()[0], "kernel_build": platform.version()}, + "config": asdict(config), "runtime": runtime, + "executable": executable, "trees": inventories, "files": files, + "native_dependencies": graph, + "aliases": alias_targets, + "trust_boundary": "exact-os-build-shared-cache-and-exclusive-host-ownership", + }) + + +def verify_native_material(raw: bytes, *, expected_digest: str, + dependency_lock: bytes) -> None: + """Compare to a digest bound by the host, never to a self-issued expectation.""" + if hashlib.sha256(raw).hexdigest() != expected_digest: + raise ValueError("native material digest differs") + value = strict_json(raw) + if (type(value) is not dict or value.get("schema") != "synaptic-native-mlx-capture/v1" + or canonical(value) != raw): + raise ValueError("native capture schema or canonical bytes differ") + runtime = value["runtime"] + expected = MLXRuntimeExpectation.from_dict({ + name: runtime[name] for name in MLXRuntimeExpectation.__dataclass_fields__ + }) + actual = capture_native_material(NativeCaptureConfig.from_dict(value["config"]), expected, + dependency_lock=dependency_lock) + if actual != raw: + raise ValueError("native runtime bytes or library closure differ") + + +def native_runtime_spec(raw: bytes): + """Represent captured native bytes in the existing compiler contract. + + Structural representation grants no execution or hardware authority. The + selected host admission must independently authenticate and revalidate the + capture; no OCI image or Modal build is fabricated. + """ + from tuner.training.contracts import RuntimeSpec + + value = strict_json(raw) + fields = { + "schema", "status", "platform", "config", "runtime", "executable", + "trees", "files", "native_dependencies", "aliases", "trust_boundary", + } + if (type(value) is not dict or set(value) != fields + or value["schema"] != "synaptic-native-mlx-capture/v1" + or value["status"] != "OBSERVED_NOT_ADMITTED" or canonical(value) != raw): + raise ValueError("native material capture is invalid") + platform_value = value["platform"] + if (type(platform_value) is not dict or set(platform_value) != {"system", "machine", "macos", "kernel_build"} + or platform_value["system"] != "Darwin" or platform_value["machine"] != "arm64" + or any(type(platform_value[key]) is not str or not platform_value[key] + for key in ("macos", "kernel_build"))): + raise ValueError("native material platform is unsupported") + NativeCaptureConfig.from_dict(value["config"]) + runtime = value["runtime"] + if type(runtime) is not dict or set(runtime) != set(MLXRuntimeExpectation.__dataclass_fields__) | {"status"}: + raise ValueError("native material runtime fields differ") + if runtime["status"] != "CANDIDATE_MATCH_ONLY": + raise ValueError("native material comparison status differs") + expected = MLXRuntimeExpectation.from_dict({ + name: runtime[name] for name in MLXRuntimeExpectation.__dataclass_fields__ + }) + return RuntimeSpec(None, expected.dependency_lock_sha256, expected.python_version, + "native_environment", hashlib.sha256(raw).hexdigest()) diff --git a/tuner/runtime/native_mlx_publication.py b/tuner/runtime/native_mlx_publication.py new file mode 100644 index 00000000..7a667e99 --- /dev/null +++ b/tuner/runtime/native_mlx_publication.py @@ -0,0 +1,223 @@ +"""Bounded native artifact projection for the existing public artifact roles. + +No ML imports, file extraction, network, database, or capability promotion. +The provider authenticates the command and terminated child before calling this +verifier. Numeric model-change proof is recomputed from the actual saved files. +""" + +from dataclasses import asdict, dataclass +import hashlib +import io +import math +from pathlib import Path +import tarfile + +from synaptic_tuner.api.v1.results import VerifiedArtifact +from tuner.runtime.native_files import NativeTree, relative_member +from tuner.runtime.native_mlx_artifacts import adapter_tensors, finite_adapter_tensors, verify_adapter_delta +from tuner.runtime.native_mlx_input import NativeModelSnapshot +from tuner.runtime.native_mlx_material import canonical, strict_json +from tuner.training.native_mlx_compilation import parse_native_workload + + +@dataclass(frozen=True) +class NativeArtifactBudget: + per_file: int + aggregate: int + + def __post_init__(self): + if (type(self.per_file) is not int or type(self.aggregate) is not int + or not 1 <= self.per_file <= self.aggregate <= 512 * 1024 * 1024): + raise ValueError("native artifact byte budget is invalid") + + +@dataclass(frozen=True) +class NativePublishedArtifact: + descriptor: VerifiedArtifact + content: bytes + + +def _archive(members, budget): + """Deterministic POSIX tar with bounded, regular, relative members only.""" + output = io.BytesIO() + with tarfile.open(fileobj=output, mode="w", format=tarfile.USTAR_FORMAT) as archive: + for name, raw in sorted(members.items()): + relative_member(name) + # Account for headers, padding and tarfile's final record padding + # before allocating the next member. + if (len(raw) > budget.per_file + or output.tell() + len(raw) + 512 + 10240 > budget.aggregate): + raise ValueError("native artifact archive exceeds budget") + info = tarfile.TarInfo(name) + info.size, info.mode, info.mtime = len(raw), 0o600, 0 + info.uid = info.gid = 0 + info.uname = info.gname = "" + archive.addfile(info, io.BytesIO(raw)) + return output.getvalue() + + +def verify_native_publication( + *, output: Path, workload: bytes, model: NativeModelSnapshot, + tokenizer: NativeModelSnapshot, tokenizer_members: tuple[str, ...], + budget: NativeArtifactBudget, +) -> tuple[NativePublishedArtifact, ...]: + """Verify exact native output and return the five portable role payloads. + + Final-model payload is an MLX adapter tar, not merged weights or PEFT. Base + repository/revision and snapshot identity travel in its manifest. Tokenizer + member selection is explicit host config, checked against its whole pinned + snapshot. Final-evaluation numbers remain trusted admitted-worker evidence; + this CPU verifier does not rerun model inference. + """ + projection = parse_native_workload(workload) + if (type(model) is not NativeModelSnapshot or type(tokenizer) is not NativeModelSnapshot + or type(budget) is not NativeArtifactBudget + or type(tokenizer_members) is not tuple or not tokenizer_members + or tokenizer_members != tuple(sorted(set(tokenizer_members)))): + raise ValueError("native artifact projection inputs differ") + profile = projection.profile + if (model.repository, model.revision, tokenizer.repository, tokenizer.revision) != ( + profile.model_ref, profile.model_revision, profile.model_ref, profile.tokenizer_revision, + ): + raise ValueError("native artifact snapshot pins differ") + with NativeTree(output, private=True) as tree, NativeTree(Path(tokenizer.root), private=True) as token_tree: + tokenizer.verify(token_tree) + # Checkpoint spelling is the pinned upstream MLX-LM trainer contract. + checkpoints = tuple(f"{step:07d}_adapters.safetensors" for step in range( + projection.controls.steps_per_save, projection.controls.iterations + 1, + projection.controls.steps_per_save, + )) + expected = tuple(sorted(( + "adapter_config.json", "adapters.safetensors", "initial-adapters.safetensors", + "input_binding.json", "native_result.json", "workload.json", *checkpoints, + ))) + if tree.members() != expected: + raise ValueError("native artifact inventory differs") + files, total = {}, 0 + for name in expected: + raw = tree.read(name, maximum=budget.per_file) + total += len(raw) + if total > budget.aggregate: + raise ValueError("native artifact aggregate exceeds budget") + files[name] = raw + if files["workload.json"] != workload: + raise ValueError("native artifact workload differs") + result = strict_json(files["native_result.json"]) + binding = strict_json(files["input_binding.json"]) + if (canonical(result) != files["native_result.json"] + or canonical(binding) != files["input_binding.json"] + or set(binding) != { + "schema", "prepared_digest", "source", "tokenizer_snapshot_digest", + "tokenization", "workload_digest", "runtime_digest", + } + or binding["schema"] != "synaptic-native-tokenization-binding/v1" + or binding["source"] != projection.source.to_dict() + or binding["tokenizer_snapshot_digest"] != tokenizer.digest + or binding["tokenization"] != projection.tokenization.to_dict() + or binding["workload_digest"] != projection.workload.fingerprint + or binding["runtime_digest"] != profile.runtime_digest): + raise ValueError("native artifact input binding differs") + prepared_digest = binding["prepared_digest"] + if (type(prepared_digest) is not str or len(prepared_digest) != 64 + or any(c not in "0123456789abcdef" for c in prepared_digest)): + raise ValueError("native prepared token digest is invalid") + if (set(result) != { + "schema", "status", "input_binding_digest", "workload_digest", + "runtime_digest", "model_snapshot_digest", "lora", "optimizer", + "completed_updates", "adapter_format", "parameter_delta", "final_validation", + } or result["schema"] != "synaptic-native-mlx-sft-result/v1" + or result["status"] != "CANDIDATE_WORKER_COMPLETE" + or result["input_binding_digest"] != hashlib.sha256(files["input_binding.json"]).hexdigest() + or result["workload_digest"] != projection.workload.fingerprint + or result["runtime_digest"] != profile.runtime_digest + or result["model_snapshot_digest"] != model.digest + or canonical(result["lora"]) != canonical(asdict(projection.lora)) + or canonical(result["optimizer"]) != canonical(asdict(projection.optimizer)) + or type(result["completed_updates"]) is not int + or result["completed_updates"] != projection.controls.iterations + or result["adapter_format"] != "mlx-lm-lora-safetensors"): + raise ValueError("native artifact result binding differs") + config = { + "fine_tune_type": "lora", "num_layers": projection.lora.layers, + "lora_parameters": { + "rank": projection.lora.rank, "scale": projection.lora.scale, + "dropout": projection.lora.dropout, "keys": list(projection.lora.targets), + }, + } + if files["adapter_config.json"] != canonical(config): + raise ValueError("native adapter configuration differs") + initial, final = files["initial-adapters.safetensors"], files["adapters.safetensors"] + tensors = adapter_tensors(initial) + shapes = tuple((tensor.name, tensor.shape) for tensor in tensors) + # Actual target realization was checked inside the admitted worker. + # Check pair coverage and explicit layer/target count independently. + names = {tensor.name for tensor in tensors} + stems = {name.rsplit(".", 1)[0] for name in names} + if (len(stems) != projection.lora.layers * len(projection.lora.targets) + or any({stem + ".lora_a", stem + ".lora_b"} - names for stem in stems) + or any(projection.lora.rank not in shape for _, shape in shapes) + or any(not any(stem.endswith("." + target) for target in projection.lora.targets) + for stem in stems)): + raise ValueError("native saved adapter target layout differs") + delta = verify_adapter_delta(initial, final, expected_shapes=shapes) + if result["parameter_delta"] != delta: + raise ValueError("native parameter delta evidence differs") + for name in checkpoints: + checkpoint = finite_adapter_tensors(files[name]) + if tuple((tensor.name, tensor.shape, tensor.dtype) for tensor in checkpoint) != tuple( + (tensor.name, tensor.shape, tensor.dtype) for tensor in tensors + ): + raise ValueError("native checkpoint tensor inventory differs") + validation = result["final_validation"] + if (type(validation) is not dict or set(validation) != { + "adapter_sha256", "phase", "loss", "rows", "supervised_tokens", + } or validation["adapter_sha256"] != hashlib.sha256(final).hexdigest() + or validation["phase"] != "reloaded_saved_final_adapter" + or type(validation["loss"]) not in (int, float) + or not math.isfinite(validation["loss"]) or validation["loss"] < 0 + or any(type(validation[key]) is not int or validation[key] < 1 + for key in ("rows", "supervised_tokens"))): + raise ValueError("native final-adapter validation differs") + model_members = { + name: files[name] for name in ("adapters.safetensors", "adapter_config.json", *checkpoints) + } + model_members["native_model.json"] = canonical({ + "schema": "synaptic-native-mlx-model/v1", "format": "mlx-lm-lora-safetensors", + "repository": model.repository, "revision": model.revision, + "snapshot_digest": model.digest, "workload_digest": projection.workload.fingerprint, + "runtime_digest": profile.runtime_digest, + }) + tokenizer_files = {} + available = {item.path for item in tokenizer.files} + for name in tokenizer_members: + if name not in available or name == "native_tokenizer.json": + raise ValueError("native tokenizer publication member differs") + raw = token_tree.read(name, maximum=budget.per_file) + total += len(raw) + if total > budget.aggregate: + raise ValueError("native tokenizer aggregate exceeds budget") + tokenizer_files[name] = raw + tokenizer_files["native_tokenizer.json"] = canonical({ + "schema": "synaptic-native-tokenizer/v1", "repository": tokenizer.repository, + "revision": tokenizer.revision, "snapshot_digest": tokenizer.digest, + "tokenization": projection.tokenization.to_dict(), + }) + roles = { + "final_model": _archive(model_members, budget), + "tokenizer": _archive(tokenizer_files, budget), + "workload_record": workload, + "training_metrics": files["native_result.json"], + "training_lineage": canonical({ + "schema": "synaptic-native-training-lineage/v1", + "source_digest": profile.source_digest, "input_binding": binding, + "model_snapshot_digest": model.digest, + }), + } + if (sum(len(raw) for raw in roles.values()) > budget.aggregate + or any(len(raw) > budget.per_file for raw in roles.values())): + raise ValueError("native public role byte budget exceeded") + token_tree.check() + tree.check() + return tuple(NativePublishedArtifact( + VerifiedArtifact(role, hashlib.sha256(raw).hexdigest(), len(raw)), raw, + ) for role, raw in sorted(roles.items())) diff --git a/tuner/runtime/native_process.py b/tuner/runtime/native_process.py new file mode 100644 index 00000000..a395a712 --- /dev/null +++ b/tuner/runtime/native_process.py @@ -0,0 +1,200 @@ +"""One-shot native process transport using the existing host record port. + +This internal transport does not replace coordinator/Foundation admission. +Claim-before-spawn prevents automatic replay after every uncertain boundary. +Only a live owner may cancel; restart reads retain uncertainty, never kill by PID. +""" + +from __future__ import annotations + +from dataclasses import asdict, dataclass +import hashlib +import os +from pathlib import Path +import re +import subprocess +import threading + +from tuner.runtime.native_files import NativeTree +from tuner.runtime.native_mlx_material import canonical, strict_json + + +@dataclass(frozen=True) +class NativeProcessLimits: + timeout_seconds: int + termination_grace_seconds: int + maximum_output_bytes: int + maximum_file_bytes: int + + def __post_init__(self): + for name, minimum, maximum in ( + ("timeout_seconds", 1, 86400), ("termination_grace_seconds", 1, 30), + ("maximum_output_bytes", 1, 16 * 1024 * 1024), + ("maximum_file_bytes", 1, 512 * 1024 * 1024), + ): + value = getattr(self, name) + if type(value) is not int or not minimum <= value <= maximum: + raise ValueError("native process bound is invalid") + + +@dataclass(frozen=True) +class NativeInvocation: + attempt: str + binding_digest: str + executable: str + executable_digest: str + guardian: str + guardian_digest: str + worker: str + worker_digest: str + request: str + request_digest: str + private_root: str + limits: NativeProcessLimits + + def __post_init__(self): + if type(self.attempt) is not str or not re.fullmatch(r"[a-zA-Z0-9_-]{1,128}", self.attempt): + raise ValueError("native attempt reference is invalid") + if type(self.limits) is not NativeProcessLimits: + raise TypeError("native process limits must be exact") + for name in ("binding_digest", "executable_digest", "guardian_digest", "worker_digest", "request_digest"): + value = getattr(self, name) + if type(value) is not str or not re.fullmatch(r"[0-9a-f]{64}", value): + raise ValueError("native process digest is invalid") + for name in ("executable", "guardian", "worker", "request", "private_root"): + value = getattr(self, name) + if (type(value) is not str or not value.startswith("/") or str(Path(value)) != value + or ".." in Path(value).parts or any(ord(c) < 32 for c in value)): + raise ValueError("native process path is invalid") + if Path(self.request).parent != Path(self.private_root): + raise ValueError("native request must be directly inside owned root") + + @property + def digest(self): + return hashlib.sha256(canonical(asdict(self))).hexdigest() + + +class NativeOwnedProcess: + """Retained single-use transport. No durable state is opened by the engine.""" + + def __init__(self, records): + self.records = records + self._live = {} + self._lock = threading.Lock() + + def _key(self, attempt, stage): + return f"native-mlx/{attempt}/{stage}" + + def _retain(self, invocation, stage, value): + raw = canonical({ + "schema": "synaptic-native-process-record/v1", + "invocation_digest": invocation.digest, "binding_digest": invocation.binding_digest, + "stage": stage, "value": value, + }) + if self.records.create(partition="effects", key=self._key(invocation.attempt, stage), canonical=raw) is not True: + raise ValueError("native process record already exists") + observed = self.records.read(partition="effects", key=self._key(invocation.attempt, stage)) + if observed is None or observed.canonical != raw: + raise ValueError("native process record readback differs") + + def observe(self, invocation): + """Known outcome only; missing acknowledgement never permits re-spawn.""" + claim = self.records.read(partition="effects", key=self._key(invocation.attempt, "claim")) + if claim is None: + return {"state": "unclaimed"} + if strict_json(claim.canonical)["invocation_digest"] != invocation.digest: + raise ValueError("native retained invocation differs") + outcome = self.records.read(partition="effects", key=self._key(invocation.attempt, "outcome")) + if outcome is None: + return {"state": "uncertain", "automatic_replay": False} + value = strict_json(outcome.canonical) + if value["invocation_digest"] != invocation.digest: + raise ValueError("native retained outcome differs") + return {"state": "terminated", "outcome": value["value"], "automatic_replay": False} + + def cancel(self, invocation): + """Signal only the current owner's pipe, never an observed numeric PID.""" + with self._lock: + item = self._live.get(invocation.attempt) + if item is None or item[0] != invocation.digest: + return False + try: + os.write(item[1], b"stop") + return True + except (BrokenPipeError, OSError): + return False + + def run_once(self, invocation: NativeInvocation): + if type(invocation) is not NativeInvocation or os.name != "posix": + raise TypeError("native invocation requires supported POSIX transport") + # Revalidation and all request/transport reads precede the durable claim. + with NativeTree(Path(invocation.private_root), private=True) as root: + for name in ("executable", "guardian", "worker", "request"): + path = Path(getattr(invocation, name)) + with NativeTree(path.parent) as source: + raw = source.read(path.name, maximum=64 * 1024 * 1024) + if hashlib.sha256(raw).hexdigest() != getattr(invocation, name + "_digest"): + raise ValueError("native process input bytes differ") + root.check() + self._retain(invocation, "claim", {"automatic_replay": False}) + read_fd, write_fd = os.pipe() + guardian = None + try: + # No inherited credentials, PYTHONPATH, DYLD_*, loader settings or + # ambient cache/home; all worker writes stay in owned private roots. + environment = { + "PATH": str(Path(invocation.executable).parent) + ":/usr/bin:/bin", + "HOME": invocation.private_root, "TMPDIR": invocation.private_root, + "HF_HUB_OFFLINE": "1", "TRANSFORMERS_OFFLINE": "1", + "TOKENIZERS_PARALLELISM": "false", + } + spec = canonical({ + "argv": [invocation.executable, "-I", "-S", "-B", invocation.worker, + "--request", invocation.request], + "cwd": invocation.private_root, "environment": environment, + **asdict(invocation.limits), + }) + b"\n" + if len(spec) > 65536: + raise ValueError("native process request exceeds guardian bound") + guardian = subprocess.Popen( + [invocation.executable, "-I", "-S", "-B", invocation.guardian, str(read_fd)], + stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, + env=environment, cwd=invocation.private_root, start_new_session=True, + close_fds=True, pass_fds=(read_fd,), + ) + os.close(read_fd) + read_fd = None + with self._lock: + self._live[invocation.attempt] = (invocation.digest, write_fd) + self._retain(invocation, "spawn", {"guardian_pid": guardian.pid, + "restart_signal_authority": False}) + # communicate starts the guardian only after spawn is retained. + timeout = invocation.limits.timeout_seconds + 2 * invocation.limits.termination_grace_seconds + 10 + stdout, _ = guardian.communicate(input=spec, timeout=timeout) + if guardian.returncode != 0 or len(stdout) > 4096: + raise ValueError("native guardian outcome unavailable") + outcome = strict_json(stdout, maximum=4096) + if (type(outcome) is not dict or outcome.get("schema") != "synaptic-native-process-outcome/v1" + or outcome.get("termination_confirmed") is not True): + raise ValueError("native worker termination is uncertain") + self._retain(invocation, "outcome", outcome) + return outcome + finally: + with self._lock: + self._live.pop(invocation.attempt, None) + # Closing our sole liveness writer makes a running guardian stop + # its worker even when an intervening store operation failed. + os.close(write_fd) + if read_fd is not None: + os.close(read_fd) + if guardian is not None: + if guardian.stdin and not guardian.stdin.closed: + guardian.stdin.close() + try: + guardian.wait(timeout=2 * invocation.limits.termination_grace_seconds + 2) + except subprocess.TimeoutExpired: + # Do not kill the guardian responsible for owned cleanup. + # Leave the claim uncertain and never auto-replay. + pass + if guardian.stdout: + guardian.stdout.close() diff --git a/tuner/runtime/native_process_guardian.py b/tuner/runtime/native_process_guardian.py new file mode 100644 index 00000000..a78fd0d0 --- /dev/null +++ b/tuner/runtime/native_process_guardian.py @@ -0,0 +1,179 @@ +"""Standalone stdlib guardian for one owned native worker, started with -I -S. + +The parent holds a pipe open. EOF (including parent death) terminates the worker +process group. A persisted PID is never reused as restart-time signal authority. +This is POSIX/Darwin process supervision, not a sandbox or Linux pidfd guarantee. +""" + +from __future__ import annotations + +import json +import os +import resource +import select +import selectors +import signal +import subprocess +import sys +import time + + +class _ExitObserver: + """Observe exit without reaping: Darwin kqueue or POSIX waitid where exposed.""" + + def __init__(self, child): + self.child = child + self.queue = None + self.done = False + if sys.platform == "darwin": + self.queue = select.kqueue() + event = select.kevent( + child.pid, filter=select.KQ_FILTER_PROC, + flags=select.KQ_EV_ADD | select.KQ_EV_ONESHOT, fflags=select.KQ_NOTE_EXIT, + ) + try: + self._events(self.queue.control([event], 1, 0)) + except BaseException: + self.queue.close() + raise + elif not hasattr(os, "waitid"): + raise ValueError("native exit observation unavailable") + + def _events(self, events): + for event in events: + if event.flags & select.KQ_EV_ERROR: + raise ValueError("native process exit watch rejected") + if event.ident != self.child.pid or not event.fflags & select.KQ_NOTE_EXIT: + raise ValueError("native process exit observation differs") + self.done = True + + def exited(self): + if self.queue is not None: + if not self.done: + self._events(self.queue.control(None, 1, 0)) + return self.done + return os.waitid(os.P_PID, self.child.pid, os.WEXITED | os.WNOHANG | os.WNOWAIT) is not None + + def close(self): + if self.queue is not None: + self.queue.close() + + +def _terminate(child, grace, observer=None): + # Do not poll/reap before signalling the still-owned group: its leader PID + # cannot be recycled while retained as our unreaped child. + try: + os.killpg(child.pid, signal.SIGTERM) + except ProcessLookupError: + pass + deadline = time.monotonic() + grace + # Both observation implementations leave the child unreaped. + while time.monotonic() < deadline: + if observer is not None: + try: + if observer.exited(): + break + except Exception: + observer = None + time.sleep(min(.02, max(0, deadline - time.monotonic()))) + try: + os.killpg(child.pid, signal.SIGKILL) + except ProcessLookupError: + pass + try: + return child.wait(timeout=grace) + except subprocess.TimeoutExpired: + return None + + +def supervise(spec, control_fd): + resource.setrlimit(resource.RLIMIT_CORE, (0, 0)) + # This limits each file; the worker/artifact inventory enforces aggregate size. + maximum_file = spec["maximum_file_bytes"] + resource.setrlimit(resource.RLIMIT_FSIZE, (maximum_file, maximum_file)) + child = None + observer = None + selector = selectors.DefaultSelector() + started = time.monotonic() + reason = "worker_exited" + counts = {"stdout": 0, "stderr": 0} + try: + child = subprocess.Popen( + spec["argv"], stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, cwd=spec["cwd"], env=spec["environment"], + start_new_session=True, close_fds=True, + ) + observer = _ExitObserver(child) + # Only the guardian owns this live handle. Parent records refer to the + # guardian; they cannot reacquire this child's cancellation authority. + for pipe, name in ((child.stdout, "stdout"), (child.stderr, "stderr")): + os.set_blocking(pipe.fileno(), False) + selector.register(pipe, selectors.EVENT_READ, name) + os.set_blocking(control_fd, False) + selector.register(control_fd, selectors.EVENT_READ, "control") + while True: + if time.monotonic() - started >= spec["timeout_seconds"]: + reason = "deadline" + break + # Observe without reaping, retaining PID ownership until group cleanup. + exited = observer.exited() + events = selector.select(.02) + for key, _ in events: + chunk = os.read(key.fd, 65536) + if key.data == "control": + # Any parent command requests cancellation; EOF means owner loss. + reason = "cancel_requested" if chunk else "owner_lost" + break + if not chunk: + selector.unregister(key.fileobj) + else: + counts[key.data] += len(chunk) + if counts[key.data] > spec["maximum_output_bytes"]: + reason = "output_limit" + break + if reason != "worker_exited": + break + if exited: + # Drain to EOF with the same limits. A descendant retaining a + # pipe cannot keep this loop alive past the overall deadline. + if all(key.data == "control" for key in selector.get_map().values()): + break + code = _terminate(child, spec["termination_grace_seconds"], observer) + return { + "schema": "synaptic-native-process-outcome/v1", + "reason": reason, "exit_code": code, "termination_confirmed": code is not None, + "stdout_bytes": counts["stdout"], "stderr_bytes": counts["stderr"], + } + finally: + selector.close() + if child is not None: + # _terminate already reaps on normal return. Do not signal a reused + # process ID after reaping; this finally only owns an unreaped child. + if child.returncode is None: + _terminate(child, spec["termination_grace_seconds"], observer) + child.stdout.close() + child.stderr.close() + if observer is not None: + observer.close() + + +def main(): + # Descriptor and JSON come only from the admitted launcher, not CLI users. + if len(sys.argv) != 2: + return 120 + control_fd = int(sys.argv[1]) + raw = sys.stdin.buffer.readline(65537) + if len(raw) > 65536 or not raw.endswith(b"\n"): + return 120 + spec = json.loads(raw) + try: + outcome = supervise(spec, control_fd) + sys.stdout.write(json.dumps(outcome, sort_keys=True, separators=(",", ":"))) + return 0 + except BaseException: + # No exception values, library stderr or credential-bearing diagnostics. + return 121 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tuner/training/contracts.py b/tuner/training/contracts.py index 85073eb4..1d210d10 100644 --- a/tuner/training/contracts.py +++ b/tuner/training/contracts.py @@ -287,10 +287,10 @@ def __post_init__(self) -> None: image = _required(self.image, "image") if _PINNED_IMAGE_PATTERN.fullmatch(image) is None or self.material_digest is not None: raise ValueError("image must be pinned to an exact sha256 digest") - elif self.material_kind == "modal_build": + elif self.material_kind in {"modal_build", "native_environment"}: if self.image is not None or type(self.material_digest) is not str \ or _SHA256_PATTERN.fullmatch(self.material_digest) is None: - raise ValueError("Modal build runtime requires only a material digest") + raise ValueError("non-OCI runtime requires only a material digest") image = None else: raise ValueError("runtime material kind is unsupported") diff --git a/tuner/training/native_mlx_authority.py b/tuner/training/native_mlx_authority.py new file mode 100644 index 00000000..99ba7f34 --- /dev/null +++ b/tuner/training/native_mlx_authority.py @@ -0,0 +1,87 @@ +"""Native reader evidence over the existing host HMAC implementation.""" + +from dataclasses import replace +import hashlib +import hmac + +from tuner.execution.coordinator_v1.model import ( + ArtifactVerificationContentV1, AuthenticatedArtifactVerificationReceiptV1, +) +from tuner.training.modal_host_authority import ReaderEvidenceAuthority +from tuner.runtime.native_mlx_material import canonical, strict_json + + +class NativeEvidenceAuthority(ReaderEvidenceAuthority): + """Host constructs keys; no key generation, secret resolution, or storage.""" + + def verification(self, content): + if type(content) is not ArtifactVerificationContentV1: + raise TypeError("exact native verification content required") + receipt = AuthenticatedArtifactVerificationReceiptV1( + content, self.authority_ref, self.key_ref, + self._tag("native-artifact-verification/v1", content.content_digest), + ) + return AuthenticatedArtifactVerificationReceiptV1.parse(receipt.canonical_bytes) + + def authenticate_verification(self, value): + try: + owned = AuthenticatedArtifactVerificationReceiptV1.parse(value.canonical_bytes) + return ( + type(value) is AuthenticatedArtifactVerificationReceiptV1 and value == owned + and owned.authority_ref == self.authority_ref and owned.key_ref == self.key_ref + and hmac.compare_digest(owned.tag, self._tag( + "native-artifact-verification/v1", owned.content.content_digest, + )) + ) + except Exception: + return False + + def process_outcome(self, invocation_digest, outcome): + content = canonical({ + "schema": "synaptic-native-terminal-evidence/v1", + "invocation_digest": invocation_digest, "outcome": outcome, + }) + return canonical({ + "content": strict_json(content), "authority_ref": self.authority_ref, "key_ref": self.key_ref, + "tag": self._tag("native-process-outcome/v1", hashlib.sha256(content).hexdigest()), + }) + + def authenticate_process_outcome(self, raw, invocation_digest): + value = strict_json(raw, maximum=8192) + if (type(value) is not dict or set(value) != {"content", "authority_ref", "key_ref", "tag"} + or canonical(value) != raw or value["authority_ref"] != self.authority_ref + or value["key_ref"] != self.key_ref + or value["content"].get("schema") != "synaptic-native-terminal-evidence/v1" + or value["content"].get("invocation_digest") != invocation_digest + or not hmac.compare_digest(value["tag"], self._tag( + "native-process-outcome/v1", hashlib.sha256(canonical(value["content"])).hexdigest(), + ))): + raise ValueError("native terminal evidence is unauthenticated") + return value["content"]["outcome"] + + +class NativeMaterialAuthority(ReaderEvidenceAuthority): + """Host-only issuer; call only after the host's source/admission checks.""" + + def issue(self, material): + from tuner.training.native_mlx_family import NativeCommandMaterialV1 + if type(material) is not NativeCommandMaterialV1: + raise TypeError("exact native command material required") + return replace( + material, authority_ref=self.authority_ref, key_ref=self.key_ref, + tag=self._tag("native-command-material/v1", hashlib.sha256(material.content_bytes()).hexdigest()), + ) + + def authenticate(self, material): + try: + from tuner.training.native_mlx_family import NativeCommandMaterialV1 + owned = NativeCommandMaterialV1.parse(material.canonical_bytes) + return ( + type(material) is NativeCommandMaterialV1 and owned == material + and owned.authority_ref == self.authority_ref and owned.key_ref == self.key_ref + and hmac.compare_digest(owned.tag, self._tag( + "native-command-material/v1", hashlib.sha256(owned.content_bytes()).hexdigest(), + )) + ) + except Exception: + return False diff --git a/tuner/training/native_mlx_compilation.py b/tuner/training/native_mlx_compilation.py new file mode 100644 index 00000000..7448e107 --- /dev/null +++ b/tuner/training/native_mlx_compilation.py @@ -0,0 +1,169 @@ +"""Pure projection of public SFT input into the internal native worker contract. + +This adds no public verb, registration, grant, or execution-mode fallback. The +host binds this compiled workload and its reviewed profile through the existing +coordinator; hardware/provenance qualification remains a separate gate. +""" + +from __future__ import annotations + +from dataclasses import asdict, dataclass +import re + +from synaptic_tuner.api.v1.training_input import TrainingInputV1, TrainingMethodV1 +from synaptic_tuner.api.v1.training_sources import PreparedTrainingInputIdentity +from tuner.runtime.native_mlx_composition import NativeAdamW, NativeLoRA +from tuner.runtime.native_mlx_input import NativeTokenization +from tuner.runtime.native_mlx_material import canonical, strict_json +from tuner.runtime.native_mlx_sft import MLXSFTConfig +from tuner.training.recipes import CompiledWorkload + + +@dataclass(frozen=True) +class NativeMLXProfile: + profile_ref: str + model_ref: str + model_revision: str + tokenizer_revision: str + runtime_digest: str + source_digest: str + lora_layers: int + lora_scale_policy: str + beta1: float + beta2: float + epsilon: float + weight_decay: float + bias_correction: bool + schedule: str + use_fast_tokenizer: bool + steps_per_report: int + steps_per_eval: int + maximum_updates: int + maximum_batch_tokens: int + + def __post_init__(self): + for value in (self.profile_ref, self.model_ref): + if type(value) is not str or not 1 <= len(value) <= 256 or any(ord(c) < 32 for c in value): + raise ValueError("native profile reference is invalid") + for value in (self.model_revision, self.tokenizer_revision): + if type(value) is not str or not re.fullmatch("[0-9a-f]{40}", value): + raise ValueError("native profile model/tokenizer pin is invalid") + for value in (self.runtime_digest, self.source_digest): + if type(value) is not str or not re.fullmatch("[0-9a-f]{64}", value): + raise ValueError("native profile material/source pin is invalid") + if self.lora_scale_policy != "alpha_over_rank": + raise ValueError("native LoRA scale conversion must be explicitly alpha_over_rank") + for value in (self.lora_layers, self.steps_per_report, self.steps_per_eval, + self.maximum_updates, self.maximum_batch_tokens): + if type(value) is not int or not 1 <= value <= 1000000: + raise ValueError("native profile bound is invalid") + if type(self.use_fast_tokenizer) is not bool: + raise ValueError("native tokenizer implementation must be explicit") + NativeAdamW(.001, self.beta1, self.beta2, self.epsilon, self.weight_decay, + self.bias_correction, self.schedule) + + @classmethod + def from_dict(cls, value): + if type(value) is not dict or set(value) != set(cls.__dataclass_fields__): + raise ValueError("native profile fields differ") + return cls(**value) + + +@dataclass(frozen=True) +class NativeSFTProjection: + workload: CompiledWorkload + source: PreparedTrainingInputIdentity + controls: MLXSFTConfig + tokenization: NativeTokenization + lora: NativeLoRA + optimizer: NativeAdamW + profile: NativeMLXProfile + + +def compile_native_sft( + training: TrainingInputV1, source: PreparedTrainingInputIdentity, profile: NativeMLXProfile, +) -> NativeSFTProjection: + if (type(training) is not TrainingInputV1 or type(source) is not PreparedTrainingInputIdentity + or type(profile) is not NativeMLXProfile or training.method is not TrainingMethodV1.SFT): + raise ValueError("native compilation requires exact public prepared SFT inputs") + # Detach mutable nested chat-template data through the existing public parser. + training = TrainingInputV1.from_json(training.canonical_json()) + if training.dataset.ref != source.ref: + raise ValueError("native public request does not bind prepared publication") + if (training.model.ref, training.model.revision, training.model.tokenizer_revision) != ( + profile.model_ref, profile.model_revision, profile.tokenizer_revision, + ): + raise ValueError("native request differs from selected model profile") + hp = training.hyperparameters + if (hp.gradient_accumulation_steps != 1 or hp.use_dora or hp.use_rslora + or hp.init_lora_weights is not True or hp.split_dataset + or hp.use_preassigned_splits is not True or hp.packing is not False + or hp.require_memory_efficient_loss is not False + or hp.duration.max_steps is None or hp.duration.num_epochs is not None): + raise ValueError("native SFT request contains unsupported controls") + if hp.duration.max_steps > profile.maximum_updates: + raise ValueError("native SFT update bound exceeded") + # Upstream retains every periodic checkpoint. Do not silently reinterpret a + # requested pruning policy or drop checkpoints claimed by the public request. + checkpoints = hp.duration.max_steps // hp.save_steps + if not training.artifacts.retain_checkpoints or checkpoints > hp.save_total_limit: + raise ValueError("native checkpoint retention policy is unsupported") + if not set(training.artifacts.required_kinds) <= { + "workload_record", "training_lineage", "training_metrics", "final_model", "tokenizer", + }: + raise ValueError("native artifact role is unsupported") + formats = { + "syntunia-sft-row/v1": "raw_text", "syntunia-sft-row/v2": "messages", + "synaptic-conversation-sft-row/v1": "messages", + } + if source.format not in formats or hp.dataset_format != formats[source.format]: + raise ValueError("native prepared format differs from public request") + if hp.dataset_format == "raw_text": + if hp.completion_only_loss or hp.assistant_only_loss or hp.prompt_render != "full_conversation": + raise ValueError("native raw text requires full sequence supervision") + mask = False + else: + if (hp.completion_only_loss is not True or hp.assistant_only_loss is not False + or hp.prompt_render != "prompt_completion"): + raise ValueError("native prepared messages require final completion supervision") + # The shared preprocessor's assistant_only_loss names this semantic + # masking operation. Public completion_only_loss is the final-target lane. + mask = True + controls = MLXSFTConfig( + hp.batch_size, hp.max_seq_length, profile.maximum_batch_tokens, + hp.duration.max_steps, hp.seed, profile.steps_per_report, profile.steps_per_eval, + hp.save_steps, + ) + tokenization = NativeTokenization( + mask, hp.prompt_render, + None if hp.chat_template_kwargs is None else canonical(hp.chat_template_kwargs), + profile.use_fast_tokenizer, + ) + lora = NativeLoRA(profile.lora_layers, hp.lora_rank, hp.lora_alpha / hp.lora_rank, + hp.lora_dropout, hp.lora_target_modules) + optimizer = NativeAdamW(hp.learning_rate, profile.beta1, profile.beta2, profile.epsilon, + profile.weight_decay, profile.bias_correction, profile.schedule) + document = { + "schema_version": "synaptic-native-mlx-workload/v1", "method": "sft", + "entrypoint": "tuner.runtime.native_mlx_child:main", + "training_input": training.to_dict(), "prepared_input": source.to_dict(), + "profile": asdict(profile), + "controls": asdict(controls), "tokenization": tokenization.to_dict(), + "lora": asdict(lora), "optimizer": asdict(optimizer), + } + workload = CompiledWorkload("sft", document["schema_version"], document["entrypoint"], canonical(document)) + return NativeSFTProjection(workload, source, controls, tokenization, lora, optimizer, profile) + + +def parse_native_workload(raw: bytes) -> NativeSFTProjection: + document = strict_json(raw, maximum=1024 * 1024) + if type(document) is not dict: + raise ValueError("native workload must be an object") + projected = compile_native_sft( + TrainingInputV1.from_dict(document["training_input"]), + PreparedTrainingInputIdentity.from_dict(document["prepared_input"]), + NativeMLXProfile.from_dict(document["profile"]), + ) + if projected.workload.canonical_bytes != raw: + raise ValueError("native workload differs from public-input compilation") + return projected diff --git a/tuner/training/native_mlx_family.py b/tuner/training/native_mlx_family.py new file mode 100644 index 00000000..abf48d50 --- /dev/null +++ b/tuner/training/native_mlx_family.py @@ -0,0 +1,452 @@ +"""Native MLX family for the existing TrainingAPI/RunsAPI coordinator. + +Host-owned authenticated catalogs bind exact commands to already prepared private +inputs and invocations. A bounded synchronous submit is intentional: no background +thread touches a host's thread-affine record store. Unknown effects never replay. +Logs/cancel/reconcile are unavailable public capabilities in this version. +""" + +from dataclasses import asdict, dataclass +import hashlib +from pathlib import Path + +from synaptic_tuner.api.v1.reference.provider_family import ProviderFamilyV1, require_methods +from tuner.execution.coordinator_v1.model import ( + ArtifactManifestV1, ArtifactVerificationContentV1, AuthenticatedArtifactVerificationReceiptV1, + EffectIntentV1, + ProviderReadPurposeV1, ProviderRunObservationContentV1, ProviderRunPhaseV1, + ProviderRunReadRequestV1, VerificationVerdictV1, +) +from tuner.execution.coordinator_v1.state_machine import _derive_foundation +from tuner.execution.foundation_v2.canonical import canonical_bytes, digest_text, safe_ref +from tuner.execution.foundation_v2.commands import parse_exact_command +from tuner.execution.foundation_v2.executors import mint_resolved_executor +from tuner.execution.foundation_v2.observations import ObservationDisposition, ProviderObservationV1 +from tuner.execution.foundation_v2.references import ProviderStageRefV1, ScopedProviderRunRefV1 +from tuner.runtime.native_files import NativeFile, NativeTree +from tuner.runtime.native_mlx_input import NativeModelSnapshot +from tuner.runtime.native_mlx_material import canonical, strict_json +from tuner.runtime.native_mlx_publication import NativeArtifactBudget, verify_native_publication +from tuner.runtime.native_process import NativeInvocation, NativeOwnedProcess, NativeProcessLimits +from tuner.training.native_mlx_planning import NativePlanningV1 + + +@dataclass(frozen=True) +class NativeCommandMaterialV1: + """Content which the host catalog's independent authority authenticates. + + The host must validate its execution-source evidence, prepared source lease, + candidate wheel/install provenance and current hardware before issuing this + binding. Neither a matching digest nor catalog presence replaces that step. + """ + + command_bytes: bytes + invocation: NativeInvocation + model: NativeModelSnapshot + tokenizer: NativeModelSnapshot + tokenizer_members: tuple[str, ...] + output: str + artifact_budget: NativeArtifactBudget + authority_ref: str + key_ref: str + tag: str + + def __post_init__(self): + if (type(self.command_bytes) is not bytes or type(self.invocation) is not NativeInvocation + or type(self.model) is not NativeModelSnapshot or type(self.tokenizer) is not NativeModelSnapshot + or type(self.artifact_budget) is not NativeArtifactBudget + or type(self.tokenizer_members) is not tuple or not self.tokenizer_members + or self.tokenizer_members != tuple(sorted(set(self.tokenizer_members))) + or any(type(name) is not str for name in self.tokenizer_members) + or type(self.output) is not str + or Path(self.output).parent != Path(self.invocation.private_root)): + raise ValueError("native command material fields are invalid") + command = parse_exact_command(self.command_bytes) + if command.digest != self.invocation.binding_digest: + raise ValueError("native invocation does not bind command") + safe_ref(self.authority_ref, "authority_ref") + safe_ref(self.key_ref, "key_ref") + digest_text(self.tag, "tag") + + def content_bytes(self): + return canonical({ + "schema": "synaptic-native-command-material/v1", + "command": strict_json(self.command_bytes), + "invocation": asdict(self.invocation), "model": asdict(self.model), + "tokenizer": asdict(self.tokenizer), "tokenizer_members": self.tokenizer_members, + "output": self.output, "artifact_budget": asdict(self.artifact_budget), + }) + + @property + def canonical_bytes(self): + return canonical({ + "content": strict_json(self.content_bytes()), "authority_ref": self.authority_ref, + "key_ref": self.key_ref, "tag": self.tag, + }) + + @classmethod + def parse(cls, raw): + value = strict_json(raw, maximum=4 * 1024 * 1024) + if type(value) is not dict or set(value) != {"content", "authority_ref", "key_ref", "tag"}: + raise ValueError("native material envelope fields differ") + content = value["content"] + if (type(content) is not dict or set(content) != { + "schema", "command", "invocation", "model", "tokenizer", + "tokenizer_members", "output", "artifact_budget", + } or content["schema"] != "synaptic-native-command-material/v1"): + raise ValueError("native material content fields differ") + def snapshot(document): + return NativeModelSnapshot( + document["repository"], document["revision"], document["root"], + tuple(NativeFile(**item) for item in document["files"]), + ) + invocation = dict(content["invocation"]) + invocation["limits"] = NativeProcessLimits(**invocation["limits"]) + result = cls( + canonical(content["command"]), NativeInvocation(**invocation), + snapshot(content["model"]), snapshot(content["tokenizer"]), + tuple(content["tokenizer_members"]), content["output"], + NativeArtifactBudget(**content["artifact_budget"]), + value["authority_ref"], value["key_ref"], value["tag"], + ) + if result.canonical_bytes != raw: + raise ValueError("native material canonical bytes differ") + return result + + +class NativeCommandCatalogV1: + """Create-only authenticated command catalog over the consumer record port.""" + + def __init__(self, records, authority): + require_methods(records, "create", "read") + require_methods(authority, "authenticate") + self.records, self.authority = records, authority + + def resolve(self, command_digest): + digest_text(command_digest, "command_digest") + stored = self.records.read(partition="effects", key="native-material/" + command_digest) + if stored is None: + raise ValueError("native command material is absent") + material = NativeCommandMaterialV1.parse(stored.canonical) + if (material.invocation.binding_digest != command_digest + or self.authority.authenticate(material) is not True): + raise ValueError("native retained command material is unauthenticated") + return material + + def publish_once(self, material): + if type(material) is not NativeCommandMaterialV1 or self.authority.authenticate(material) is not True: + raise ValueError("native command material is unauthenticated") + raw = material.canonical_bytes + self.records.create(partition="effects", key="native-material/" + material.invocation.binding_digest, + canonical=raw) + retained = self.resolve(material.invocation.binding_digest) + if retained.canonical_bytes != raw: + raise ValueError("native command material collision") + return retained + + +class _NativeFamily: + effect_kinds = ("stage", "submit") + payload_schemas = ("stage-payload/v2", "submit-payload/v2") + + def __init__(self, *, planning, catalog, material_authority, records, + evidence_authority, foundation_authenticator, assessment_authenticator, clock): + if type(planning) is not NativePlanningV1: + raise TypeError("exact native planning required") + for port, methods in ( + (catalog, ("resolve",)), (material_authority, ("authenticate",)), + (evidence_authority, ("observation", "verification", "authenticate_observation", + "authenticate_verification", "authenticate_log_page", + "process_outcome", "authenticate_process_outcome")), + ): + require_methods(port, *methods) + self.planning, self.catalog, self.material_authority = planning, catalog, material_authority + self.transport = NativeOwnedProcess(records) + self.records = records + self.authority, self.foundation, self.assessments, self.clock = ( + evidence_authority, foundation_authenticator, assessment_authenticator, clock, + ) + self.descriptor = planning.executor_descriptor + self.provider_id, self.profile_ref = planning.provider.provider_id, planning.provider.profile_ref + self.account_ref, self.namespace_ref = planning.scope.account_ref, planning.scope.namespace_ref + + def _material(self, command_digest): + material = self.catalog.resolve(command_digest) + if (type(material) is not NativeCommandMaterialV1 + or self.material_authority.authenticate(material) is not True + or type(material.invocation) is not NativeInvocation + or type(material.model) is not NativeModelSnapshot + or type(material.tokenizer) is not NativeModelSnapshot + or type(material.artifact_budget) is not NativeArtifactBudget): + raise ValueError("native_command_material_unauthenticated") + command = parse_exact_command(material.command_bytes) + self.planning.validate_preparation(command.preparation) + invocation = material.invocation + if (command.digest != command_digest or invocation.binding_digest != command_digest + or invocation.limits != self.planning.limits + or command.executor != self.descriptor + or command.operation.effect.kind.value not in self.effect_kinds + or Path(material.output).parent != Path(invocation.private_root)): + raise ValueError("native_command_binding_differs") + with NativeTree(Path(invocation.private_root), private=True) as root: + raw = root.read(Path(invocation.request).name, maximum=2 * 1024 * 1024) + request = strict_json(raw) + if (hashlib.sha256(raw).hexdigest() != invocation.request_digest + or canonical(request) != raw + or canonical(request["workload"]) != self.planning.workload + or canonical(request["model_snapshot"]) != canonical(asdict(material.model)) + or canonical(request["tokenizer_snapshot"]) != canonical(asdict(material.tokenizer)) + or request["output"] != material.output): + raise ValueError("native_request_material_differs") + for key in ("runtime_capture", "dependency_lock", "dataset", "output"): + if type(request[key]) is not str or Path(request[key]).parent != root.path: + raise ValueError("native_private_material_root_differs") + capture = root.read(Path(request["runtime_capture"]).name, maximum=16 * 1024 * 1024) + if capture != self.planning.runtime_capture: + raise ValueError("native_runtime_capture_differs") + runtime = strict_json(capture) + if (invocation.executable != runtime["executable"] + or invocation.executable_digest != runtime["runtime"]["executable_sha256"]): + raise ValueError("native_interpreter_binding_differs") + lock = root.read(Path(request["dependency_lock"]).name, maximum=1024 * 1024) + if hashlib.sha256(lock).hexdigest() != runtime["runtime"]["dependency_lock_sha256"]: + raise ValueError("native_dependency_lock_differs") + source = request["workload"]["prepared_input"] + dataset = root.read(Path(request["dataset"]).name, maximum=64 * 1024 * 1024) + if len(dataset) != source["size_bytes"] or hashlib.sha256(dataset).hexdigest() != source["content_digest"]: + raise ValueError("native_prepared_source_differs") + members = { + str(Path(tree["root"]) / item["path"]): item["sha256"] + for tree in runtime["trees"] for item in tree["members"] + } + for role, suffix in ( + ("worker", "/tuner/runtime/native_mlx_child.py"), + ("guardian", "/tuner/runtime/native_process_guardian.py"), + ): + path = getattr(invocation, role) + if (not path.endswith(suffix) + or members.get(path) != getattr(invocation, role + "_digest")): + raise ValueError("native_worker_source_binding_differs") + return material, command + + def execute_once(self, payload, request): + self.planning.require_qualified() + material, command = self._material(request.command_digest) + expected_payload = self.planning.payload(command.preparation, command.operation.effect.kind) + if (payload != expected_payload or request.descriptor_digest != self.descriptor.digest + or (request.provider_id, request.profile_ref, request.account_ref, request.namespace_ref) + != (self.provider_id, self.profile_ref, self.account_ref, self.namespace_ref) + or request.effect_kind != command.operation.effect.kind.value + or request.payload_schema != payload.payload_kind or request.input_digest != payload.input_digest): + raise ValueError("native_resolution_binding_differs") + base = dict( + effect_id=command.operation.effect.effect_id, command_digest=command.digest, + executor_descriptor_digest=self.descriptor.digest, resolution_digest=request.digest, + result_epoch=1, + ) + if request.effect_kind == "stage": + # Staging admits already prepared immutable host inputs. It performs + # no install, network access, model load or provider resource creation. + return ProviderObservationV1( + **base, disposition=ObservationDisposition.FOUND, + stage_ref=ProviderStageRefV1( + self.provider_id, self.profile_ref, self.account_ref, self.namespace_ref, + "native-stage-" + command.digest, + ), + ) + try: + outcome = self.transport.run_once(material.invocation) + terminal = self.authority.process_outcome(material.invocation.digest, outcome) + key = "native-terminal/" + material.invocation.digest + if self.records.create(partition="effects", key=key, canonical=terminal) is not True: + raise ValueError("native_terminal_evidence_collision") + retained = self.records.read(partition="effects", key=key) + if (retained is None or retained.canonical != terminal + or self.authority.authenticate_process_outcome(terminal, material.invocation.digest) != outcome): + raise ValueError("native_terminal_evidence_readback_differs") + except Exception: + # Any failure after/around claim remains indeterminate. No recovery + # or cancellation authority is inferred from a saved PID. + return ProviderObservationV1(**base, disposition=ObservationDisposition.INDETERMINATE) + return ProviderObservationV1( + **base, disposition=ObservationDisposition.FOUND, + provider_run=self._run_ref(material), + ) + + def _run_ref(self, material): + return ScopedProviderRunRefV1( + self.provider_id, self.profile_ref, self.account_ref, self.namespace_ref, + "native-run-" + material.invocation.digest, + ) + + def _read(self, request, purpose): + if type(request) is not ProviderRunReadRequestV1 or request.purpose is not purpose: + raise ValueError("native_read_request_differs") + # Reconstruct the sealed read request, including its retained exact + # Foundation assessment. Never mint a fresh timestamped assessment. + ProviderRunReadRequestV1(**{ + name: getattr(request, name) for name in request.__dataclass_fields__ + }) + record = request.foundation_record + if (self.foundation.authenticate_grant(record.grant, record.command_bytes) is not True + or any(self.foundation.authenticate_receipt(value) is not True for value in record.results) + or any(self.foundation.authenticate_invalid_evidence(value) is not True + for value in record.invalid_evidence)): + raise ValueError("native_foundation_evidence_invalid") + assessment = type(request.assessment).parse(request.assessment.canonical_bytes) + if (assessment != request.assessment or self.assessments.authenticate(assessment) is not True + or assessment.content.foundation_record_digest != record.record_digest + or assessment.content.authenticated_receipt_digests + != tuple(item.authenticated_receipt_digest for item in record.results)): + raise ValueError("native_foundation_assessment_invalid") + binding, outcome, bound = _derive_foundation( + EffectIntentV1.from_command_bytes(request.submit_command_bytes), + record, assessment, self.foundation, self.assessments, request.provider_run, + ) + if (binding != request.foundation_binding or outcome != request.foundation_outcome + or bound != request.provider_run + or bound.authenticated_receipt_digest != request.found_receipt_digest): + raise ValueError("native_foundation_read_derivation_differs") + command = parse_exact_command(request.submit_command_bytes) + material, owned = self._material(command.digest) + if (owned.canonical_bytes != command.canonical_bytes + or command.operation.effect.kind.value != "submit" + or request.run != self.planning.run + or request.provider_run.reference != self._run_ref(material)): + raise ValueError("native_known_run_binding_differs") + return material + + def _outcome(self, material): + state = self.transport.observe(material.invocation) + if state["state"] != "terminated": + raise ValueError("native_termination_unconfirmed") + outcome = state["outcome"] + terminal = self.records.read(partition="effects", key="native-terminal/" + material.invocation.digest) + if (terminal is None or self.authority.authenticate_process_outcome( + terminal.canonical, material.invocation.digest) != outcome): + raise ValueError("native_terminal_evidence_differs") + if outcome.get("termination_confirmed") is not True: + raise ValueError("native_termination_unconfirmed") + return outcome + + def observe(self, request): + material = self._read(request, ProviderReadPurposeV1.OBSERVE) + outcome = self._outcome(material) + succeeded = outcome["reason"] == "worker_exited" and outcome["exit_code"] == 0 + phase = ProviderRunPhaseV1.SUCCEEDED if succeeded else ProviderRunPhaseV1.FAILED + ref = self._run_ref(material) + content = ProviderRunObservationContentV1( + "synaptic-provider-run-observation-content/v1", request.request_digest, + request.source_workflow_record_digest, request.source_revision, request.run, + request.provider_run.binding_digest, ref.provider_id, ref.profile_ref, + ref.account_ref, ref.namespace_ref, ref.provider_job_ref, phase, + canonical_bytes({"invocation_digest": material.invocation.digest, "outcome": outcome}), + None if succeeded else "native_worker_failed", "native-mlx-reader", "1.0.0", self.clock.now(), + ) + return self.authority.observation(content) + + def _artifacts(self, material): + outcome = self._outcome(material) + if outcome["reason"] != "worker_exited" or outcome["exit_code"] != 0: + raise ValueError("native_worker_did_not_succeed") + return verify_native_publication( + output=Path(material.output), workload=self.planning.workload, + model=material.model, tokenizer=material.tokenizer, + tokenizer_members=material.tokenizer_members, budget=material.artifact_budget, + ) + + def _manifest(self, material, artifacts): + return ArtifactManifestV1.build( + run=self.planning.run, provider_run=self._run_ref(material), + artifacts=tuple(item.descriptor for item in artifacts), + artifact_source_digest=material.invocation.digest, + canonical_evidence=canonical_bytes({ + "format": "mlx-lm-lora-safetensors", "invocation_digest": material.invocation.digest, + "command_digest": material.invocation.binding_digest, + }), + ) + + def artifacts(self, request): + material = self._read(request, ProviderReadPurposeV1.ARTIFACTS) + return self._manifest(material, self._artifacts(material)) + + def iter_artifact_bytes(self, request, manifest, role, *, maximum_bytes): + material = self._read(request, ProviderReadPurposeV1.ARTIFACTS) + artifacts = self._artifacts(material) + if manifest != self._manifest(material, artifacts): + raise ValueError("native_artifact_manifest_changed") + matches = [item for item in artifacts if item.descriptor.role == role] + if (len(matches) != 1 or type(maximum_bytes) is not int + or maximum_bytes < matches[0].descriptor.size_bytes): + raise ValueError("native_artifact_stream_bound_invalid") + raw = matches[0].content + for offset in range(0, len(raw), 1024 * 1024): + yield raw[offset:offset + 1024 * 1024] + + def verify(self, workflow, manifest): + command = parse_exact_command(workflow.submit.canonical_command_bytes) + material, _ = self._material(command.digest) + if (workflow.run != self.planning.run + or workflow.provider_run_ref.reference != self._run_ref(material) + or manifest != self._manifest(material, self._artifacts(material))): + raise ValueError("native_artifact_verification_binding_differs") + content = ArtifactVerificationContentV1( + "synaptic-artifact-verification-content/v1", workflow.record_digest, + workflow.revision, workflow.run, workflow.provider_run_ref.binding_digest, + manifest.manifest_digest, manifest.artifact_source_digest, manifest.artifacts, + manifest.artifacts, VerificationVerdictV1.VERIFIED, None, + "native-mlx-artifact-verifier", "1.0.0", + canonical_bytes({"invocation_digest": material.invocation.digest}), self.clock.now(), + ) + receipt = self.authority.verification(content) + if type(receipt) is not AuthenticatedArtifactVerificationReceiptV1 or receipt.content != content: + raise ValueError("native_artifact_receipt_differs") + return receipt + + def replay(self, workflow, manifest, prior_receipt): + if self.authenticate(prior_receipt) is not True: + raise ValueError("native_prior_verification_unauthenticated") + return self.verify(workflow, manifest) + + def authenticate(self, value): + return self.authority.authenticate_verification(value) + + def logs(self, *args, **kwargs): + raise ValueError("native_logs_unavailable") + + +class _ExecutorResolver: + def __init__(self, executor): + self.executor = executor + + def resolve(self, request): + return mint_resolved_executor(request, self.executor) + + +class _NoReconciliation: + def resolve(self, request): + raise ValueError("native_reconciliation_unavailable") + + +class _Authenticator: + def __init__(self, method): + self.method = method + + def authenticate(self, value): + return self.method(value) + + +def native_mlx_family(**ports): + """Compose candidate/qualified native adapters without global registration. + + Consumer uses this result with ``compose_reference_host`` and its existing + request/grant/store ports. Host ``GrantAuthorityPort.bind`` retains the + independently authenticated ``NativeCommandMaterialV1`` for each command. + """ + internal = _NativeFamily(**ports) + return ProviderFamilyV1( + internal.planning.descriptor, internal.planning, internal.planning, internal, + _ExecutorResolver(internal), _NoReconciliation(), internal.authority, internal, + _Authenticator(internal.authority.authenticate_observation), + _Authenticator(internal.authority.authenticate_log_page), + ) diff --git a/tuner/training/native_mlx_planning.py b/tuner/training/native_mlx_planning.py new file mode 100644 index 00000000..1bf366b4 --- /dev/null +++ b/tuner/training/native_mlx_planning.py @@ -0,0 +1,212 @@ +"""Native planning/preparation adapters for the existing public coordinator. + +The consumer supplies a resolved request and independently authenticated native +qualification. This module grants neither source nor hardware authority and +does not register a provider. Candidate planning remains inspectable when the +qualification port refuses execution. +""" + +from dataclasses import asdict, dataclass +from datetime import datetime, timedelta, timezone +import hashlib + +from synaptic_tuner.api.v1.planning import ( + ProviderPlanContextV1, ProviderPlanRef, ResolvedTrainingRequest, + TrainingPlan, TrainingPlanBasisV1, +) +from synaptic_tuner.api.v1.providers import ProviderCapabilities, ProviderDescriptor, ProviderRef +from synaptic_tuner.api.v1.results import TrainingRunRef +from synaptic_tuner.api.v1.training_facade import AuthorizationRequirement, TrainingPreflight +from tuner.execution.coordinator_v1.model import ProviderExecutionBindingV1 +from tuner.execution.foundation_v2.canonical import digest_text, safe_ref +from tuner.execution.foundation_v2.commands import CanonicalProviderPayloadV1 +from tuner.execution.foundation_v2.executors import AdapterDescriptorV1, ExecutorDescriptorV1 +from tuner.execution.foundation_v2.identities import EffectKind +from tuner.execution.foundation_v2.preparation import CanonicalPreparationV2 +from tuner.execution.foundation_v2.references import ExecutionScopeV1 +from tuner.runtime.native_mlx_material import canonical, native_runtime_spec +from tuner.runtime.native_process import NativeProcessLimits +from tuner.training.coordinator_material import _domain, _runtime +from tuner.training.native_mlx_compilation import parse_native_workload + + +def _digest(value): + return hashlib.sha256(canonical(value)).hexdigest() + + +@dataclass(frozen=True) +class NativeQualificationV1: + """Evidence references, authenticated by the consumer's independent port. + + These hashes are not proof by themselves. The authority must verify retained + hardware runs, source review, selected wheel/install provenance and the exact + final-adapter/public-readback evidence. No default accepting authority exists. + """ + + profile_digest: str + source_digest: str + runtime_digest: str + hardware_digest: str + attempt_evidence: tuple[str, str] + authority_ref: str + key_ref: str + tag: str + + def __post_init__(self): + for name in ("profile_digest", "source_digest", "runtime_digest", "hardware_digest", "tag"): + digest_text(getattr(self, name), name) + if (type(self.attempt_evidence) is not tuple or len(self.attempt_evidence) != 2 + or len(set(self.attempt_evidence)) != 2): + raise ValueError("two distinct native qualification attempts required") + for value in self.attempt_evidence: + digest_text(value, "attempt_evidence") + safe_ref(self.authority_ref, "authority_ref") + safe_ref(self.key_ref, "key_ref") + + +class NativePlanningV1: + """One immutable native material selection; reusable across host restarts.""" + + __slots__ = ( + "resolved", "run", "workload", "runtime_capture", "limits", "scope", + "provider", "profile_digest", "_clock", "_qualification", + "_qualification_authority", "descriptor", "executor_descriptor", + "adapter_descriptor", "binding", "context_value", "plan", "_sealed", + ) + + def __setattr__(self, name, value): + if getattr(self, "_sealed", False): + raise AttributeError("native planning selection is immutable") + object.__setattr__(self, name, value) + + def __init__( + self, *, resolved: ResolvedTrainingRequest, run: TrainingRunRef, + workload: bytes, runtime_capture: bytes, limits: NativeProcessLimits, + scope: ExecutionScopeV1, clock, qualification=None, qualification_authority=None, + ): + if (type(resolved) is not ResolvedTrainingRequest or type(run) is not TrainingRunRef + or type(limits) is not NativeProcessLimits or type(scope) is not ExecutionScopeV1): + raise TypeError("exact native plan inputs required") + projection = parse_native_workload(workload) + runtime = native_runtime_spec(runtime_capture) + if (run.project_ref != resolved.project_ref + or projection.workload.fingerprint != resolved.workload_digest + or projection.profile.source_digest != resolved.source_digest + or projection.profile.runtime_digest != runtime.material_digest + or _domain("synaptic-coordinator-runtime/v1", _runtime(runtime)) != resolved.runtime_digest + or _domain("synaptic-coordinator-artifact-policy/v1", + projection.workload.document["training_input"]["artifacts"]) + != resolved.artifact_policy_digest): + raise ValueError("native resolved request differs from compiled material") + self.resolved, self.run, self.workload = resolved, run, workload + self.runtime_capture, self.limits, self.scope = runtime_capture, limits, scope + self.provider = ProviderRef("native-mlx", projection.profile.profile_ref) + self.profile_digest = _digest({ + "schema": "synaptic-native-provider-profile/v1", + "profile": asdict(projection.profile), "limits": asdict(limits), + "scope": scope.to_dict(), + }) + self._clock = clock + self._qualification, self._qualification_authority = qualification, qualification_authority + qualified = self.qualified() + self.descriptor = ProviderDescriptor( + "synaptic-provider-descriptor/v1", "native-mlx", "Apple native MLX", "1.0.0", + ProviderCapabilities(qualified, False, False, False, qualified, False), + ) + self.executor_descriptor = ExecutorDescriptorV1("native-mlx", "owned-native-process", "1.0.0") + self.adapter_descriptor = AdapterDescriptorV1("native-mlx", "no-replay-native-reconciliation", "1.0.0") + self.binding = ProviderExecutionBindingV1( + self.provider, self.descriptor.descriptor_digest, self.profile_digest, scope, + self.executor_descriptor, self.adapter_descriptor.digest, _digest(asdict(limits)), + _digest({"schema": "synaptic-native-local-quote/v1", "amount": 0, "currency": "USD"}), + _digest({"schema": "synaptic-native-secret-requirements/v1", "secrets": []}), + ) + self.context_value = ProviderPlanContextV1( + "synaptic-provider-plan-context/v1", self.provider, + TrainingPlanBasisV1.from_resolved(resolved).basis_digest, + self.descriptor.descriptor_digest, self.profile_digest, + ) + self.plan = TrainingPlan( + "synaptic-training-plan/v2", TrainingPlanBasisV1.from_resolved(resolved), + ProviderPlanRef(self.context_value.provider_context_digest), + ) + self._sealed = True + + def qualified(self): + proof = self._qualification + try: + return ( + type(proof) is NativeQualificationV1 + and proof.profile_digest == self.profile_digest + and proof.source_digest == self.resolved.source_digest + and proof.runtime_digest == self.resolved.runtime_digest + and self._qualification_authority.authenticate(proof) is True + ) + except Exception: + return False + + def require_qualified(self): + if not self.qualified(): + raise ValueError("native_hardware_qualification_required") + + def describe(self, provider): + if provider != self.provider: + raise ValueError("native provider differs") + return self.descriptor + + def context(self, resolved, provider): + if resolved != self.resolved or provider != self.provider: + raise ValueError("native resolved material differs") + return self.context_value + + def resolve(self, provider, context): + if provider != self.provider or context != self.context_value: + raise ValueError("native context differs") + return self.binding + + def for_plan(self, plan): + if plan != self.plan: + raise ValueError("native run plan differs") + return self.run + + def preflight(self, plan): + if plan != self.plan: + raise ValueError("native plan differs") + now = self._clock.now() + expiry = (datetime.fromisoformat(now.replace("Z", "+00:00")) + timedelta(minutes=15)) + ready = self.qualified() + return TrainingPreflight( + plan.plan_fingerprint, ready, now, + expiry.astimezone(timezone.utc).isoformat().replace("+00:00", "Z"), + (AuthorizationRequirement("training.start", False),), + () if ready else ("native_hardware_qualification_required",), + ) + + def prepare(self, plan, run, binding): + if plan != self.plan or run != self.run or binding != self.binding: + raise ValueError("native preparation differs") + return CanonicalPreparationV2.build( + provider=self.provider, scope=self.scope, project_ref=run.project_ref, + run_id=run.run_id, plan_fingerprint=plan.plan_fingerprint, + source_digest=self.resolved.source_digest, workload_digest=self.resolved.workload_digest, + runtime_digest=self.resolved.runtime_digest, resource_digest=binding.resource_digest, + artifact_contract_digest=self.resolved.artifact_policy_digest, + quote_digest=binding.quote_digest, + secret_requirements_digest=binding.secret_requirements_digest, + execution_binding_digest=binding.binding_digest, + ) + + def validate_preparation(self, preparation): + expected = self.prepare(self.plan, self.run, self.binding) + if (type(preparation) is not CanonicalPreparationV2 + or preparation.canonical_bytes != expected.canonical_bytes): + raise ValueError("native preparation is not the selected immutable material") + return expected + + def payload(self, preparation, kind): + self.validate_preparation(preparation) + if kind not in (EffectKind.STAGE, EffectKind.SUBMIT): + raise ValueError("native lifecycle operation is unavailable") + return CanonicalProviderPayloadV1.build( + "native-mlx", f"{kind.value}-payload/v2", self.resolved.workload_digest, + )