Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .agents/skills/fine-tuning/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ Train language models with SFT, KTO, and GRPO locally or on supported cloud prov
| Prepare verified raw-text SFT data | `python tuner.py prepare-dataset --config <config.json> --json` |
| Check SFT loss masking (tokenizer only) | `python tuner.py doctor sft-mask --dataset-path <jsonl> --model <tokenizer>` |
| Inspect native training host candidate | `python tuner.py doctor native-training --json` — read-only platform/Python/package metadata; see `reference/native-local-training.md` |
| Diagnose an offline SFT worker rejection | Read the exact retained `trainer.stderr.log`; closed phase/category/milestone labels locate a boundary, not a cause or retry authority; see `reference/sft-training.md` |
| Native MLX prepared SFT candidate | Internal exact-token batch/loss and runtime comparison only; no native launch CLI or API provider yet; see `reference/native-local-training.md` |
| Plan a derived training image | `python scripts/qualify_derived_training_image.py plan --config Trainers/image_profiles/<profile>.yaml` |
| Warm Space scaffold | `python3 Trainers/cloud/scripts/manage_space.py render --template vllm_warm --output-dir /tmp/my-space --base-image ghcr.io/<org>/<image>:<tag>` |
| Warm Space deploy | `python3 Trainers/cloud/scripts/manage_space.py deploy --space-id <user>/<space> --template vllm_warm --base-image ghcr.io/<org>/<image>:<tag> --hardware a10g-small --sleep-time 3600 --var BASE_MODEL=<model>` |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ model:
runtime:
provider: modal
lock_path: tuner/execution/providers/modal/modal-runtime-v1.lock.json
expected_lock_sha256: 42c25ecab7848fd5002bc27711e342d1b8fac0761621af65cba3736ad9110727
expected_lock_sha256: 4f3fa81c9aeb9130258dec23f2b841e6dbdf2d686d1cf6b50249e743c3d496ca
input:
jsonl_path: /replace/with/private/prepared-dataset.jsonl
mode: text
Expand Down
8 changes: 8 additions & 0 deletions .agents/skills/fine-tuning/reference/grpo-training.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,14 @@ end-to-end provider deadline or public cancellation qualification. The first
executable candidate requires identical model and tokenizer revisions because
both consume one verified local snapshot.

The packaged runtime resolves `GRPOConfig` precision before loading the model.
When the retained model dtype is `null`, it passes the effective FP16 or BF16
dtype to Unsloth's loader. Pinned Unsloth checks that loaded dtype against
trainer precision during `GRPOTrainer` construction; allowing the loader to
auto-select BF16 on BF16-capable hardware while the recipe requests FP16 can
fail at `TRAINER_SETUP`. A closed `TypeError` marker alone cannot establish
that this was the historical cause, and it does not authorize replay.

GRPO has a separate fixed 131-member offline inventory and 13-member outer
closure. Existing SFT inventories remain unchanged. Maintenance uses the
checked-in command, offline first because the outer closure binds its bytes:
Expand Down
38 changes: 38 additions & 0 deletions .agents/skills/fine-tuning/reference/native-local-training.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,41 @@ Resolve with the documented platform/interpreter/index and install only in a
fresh private environment with hashes required and source builds disabled.
This candidate does not select a model, enable a provider, or qualify Metal,
SFT or GRPO. Keep CUDA/Modal environments and the legacy Mac checkout unchanged.

## Prepared SFT worker candidate

`tuner/runtime/native_mlx_sft.py` is an internal CPU-tested boundary. It consumes
the existing shared `PreparedSFTExample` objects after host verification and
tokenization; it does not introduce a dataset format or public execution API.
Pass the exact prepared-input identity, tokenizer revision, retained row IDs,
declared train/validation tuples and explicit `MLXSFTConfig` controls. The
projection copies all tokens/labels immutably and records a deterministic digest.
The host must still bind those rows to the authenticated source and tokenizer.

No MLX text/chat loader, extra EOS, resplit, truncation, packing, tail dropping,
or single-offset masking is allowed. Failed masking diagnostics, already
truncated rows, attention masks other than all-ones unpadded causal rows, and
rows without shifted target supervision reject. Arbitrary ignored token spans
remain ignored. Training shuffles deterministically within its own split;
validation retains every row in declared order, including partial batches.

The custom hooks target pinned MLX-LM 0.32.2. One batch is one optimizer update;
gradient accumulation and checkpointing are deliberately unavailable in this
increment. Upstream accumulation averages microbatch means, which needs a
separate objective review for unequal supervised-token counts. Model, LoRA,
optimizer and learning-rate policy must be prepared explicitly by the future
child composition; this worker does not infer or silently apply those controls.

`tuner/runtime/native_mlx_runtime.py` compares host-supplied exact interpreter
and complete distribution pins against the running candidate before worker ML
imports. `CANDIDATE_MATCH_ONLY` means metadata and executable bytes match.
It does not authenticate wheel files, imported module origins, Homebrew's linked
Python framework, other native libraries, Metal, or launch authority. Retain the
original hash lock and use the existing `uv pip freeze`, `uv pip check`,
`shasum -a 256` and `otool -L` tools for review evidence; do not install packages
into the shared interpreter or treat a newly observed inventory as approved.

There is no native launch CLI yet. The internal worker may run only in an
independently admitted, host-owned bounded child. The desktop plan lists the
missing native runtime material, child lifecycle, artifact roles and API joins.
Do not call the worker as an ad hoc substitute for that composition.
42 changes: 42 additions & 0 deletions .agents/skills/fine-tuning/reference/sft-training.md
Original file line number Diff line number Diff line change
Expand Up @@ -460,3 +460,45 @@ The CPU `probe-filesystem` command measures actual native writer modes and host
readability under a reviewed finite container plan; it is not training, GPU,
provider or cancellation qualification. Source inventory and five-role native
verification remain closed; no default capability or UID policy is promoted.

Before short-lived source preparation, run `model-preflight` with the exact
interpreter selected for `prepare-model`; the base consumer may lack the optional
Hub SDK. This checks only the existing SDK signature contract and creates no
client or claim. `prepare-model` repeats it before its one-use claim. Use an
independently verified helper environment; do not upgrade the pinned trainer.
Create and verify necessary private source/artifact parent directories first.
Preserve any consumed model-preparation claim after failure. New failures retain
the existing allowlisted `model_preparation_stage`; never infer that stage or a
cause for an older generic diagnostic, or replay an uncertain attempt.

### Offline SFT worker rejection

The isolated source worker retains exit code 2 on failure and emits one bounded
diagnostic line to the existing `tracking/trainer.stderr.log`:
`OFFLINE_SFT_WORKER_REJECTED phase=<closed> category=<closed>`.
The phases are `BOOTSTRAP`, `CLOSURE`, `BOOTSTRAP_ORIGIN`, `ARGUMENTS`,
`IMPORT_GUARD`, `TRAINER`, and `POSTCHECK`. Categories are `REJECTION`, `IMPORT`,
`OS`, `VALUE`, `SYSTEM_EXIT`, and `OTHER`. During `TRAINER`, the last valid
existing packaged-trainer hook may append `milestone=<closed>` (for example
`TORCH_IMPORT` or `MODEL_LIBRARY_LOAD`). Unknown hooks clear that milestone.
The fallback remains the bare rejection marker. This diagnostic does not print
exception text, arguments, paths, environment values, or traceback frames;
library-generated stdout/stderr is a separate surface.

Use the labels to choose the next targeted reproduction. An import category
does not identify the missing library, and a model-load milestone does not
establish OOM. Do not interpret an older bare marker retrospectively. Preserve
the consumed claim, roots, container and retained logs; the diagnostics never
authorize replay. See `docs/review/offline-sft-worker-diagnostics.md` for the
October 20 evidence boundary, CPU regressions and fresh-run prerequisites.

After an intentional worker source change, check and refresh existing
manifests with the checked-in maintainers in dependency order:
`regenerate_offline_sft_worker_closure.py`,
`regenerate_packaged_training_worker_closure.py`,
`regenerate_grpo_worker_closures.py --kind offline`, then `--kind outer`,
`regenerate_hf_parent_closure.py`, `regenerate_modal_runtime_lock.py`, and
`regenerate_modal_inference_lock.py` (all under `scripts/`). Run each read-only
check first, review its drift, use `--write` only for intentional hash refresh,
then recheck. Preserve inventories and non-hash pins. Sync canonical skills
with `.skills/scripts/sync_skill_trees.py` and verify its `--check`.
2 changes: 2 additions & 0 deletions .claude/skills/fine-tuning/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ Train language models with SFT, KTO, and GRPO locally or on supported cloud prov
| Prepare verified raw-text SFT data | `python tuner.py prepare-dataset --config <config.json> --json` |
| Check SFT loss masking (tokenizer only) | `python tuner.py doctor sft-mask --dataset-path <jsonl> --model <tokenizer>` |
| Inspect native training host candidate | `python tuner.py doctor native-training --json` — read-only platform/Python/package metadata; see `reference/native-local-training.md` |
| Diagnose an offline SFT worker rejection | Read the exact retained `trainer.stderr.log`; closed phase/category/milestone labels locate a boundary, not a cause or retry authority; see `reference/sft-training.md` |
| Native MLX prepared SFT candidate | Internal exact-token batch/loss and runtime comparison only; no native launch CLI or API provider yet; see `reference/native-local-training.md` |
| Plan a derived training image | `python scripts/qualify_derived_training_image.py plan --config Trainers/image_profiles/<profile>.yaml` |
| Warm Space scaffold | `python3 Trainers/cloud/scripts/manage_space.py render --template vllm_warm --output-dir /tmp/my-space --base-image ghcr.io/<org>/<image>:<tag>` |
| Warm Space deploy | `python3 Trainers/cloud/scripts/manage_space.py deploy --space-id <user>/<space> --template vllm_warm --base-image ghcr.io/<org>/<image>:<tag> --hardware a10g-small --sleep-time 3600 --var BASE_MODEL=<model>` |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ model:
runtime:
provider: modal
lock_path: tuner/execution/providers/modal/modal-runtime-v1.lock.json
expected_lock_sha256: 42c25ecab7848fd5002bc27711e342d1b8fac0761621af65cba3736ad9110727
expected_lock_sha256: 4f3fa81c9aeb9130258dec23f2b841e6dbdf2d686d1cf6b50249e743c3d496ca
input:
jsonl_path: /replace/with/private/prepared-dataset.jsonl
mode: text
Expand Down
8 changes: 8 additions & 0 deletions .claude/skills/fine-tuning/reference/grpo-training.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,14 @@ end-to-end provider deadline or public cancellation qualification. The first
executable candidate requires identical model and tokenizer revisions because
both consume one verified local snapshot.

The packaged runtime resolves `GRPOConfig` precision before loading the model.
When the retained model dtype is `null`, it passes the effective FP16 or BF16
dtype to Unsloth's loader. Pinned Unsloth checks that loaded dtype against
trainer precision during `GRPOTrainer` construction; allowing the loader to
auto-select BF16 on BF16-capable hardware while the recipe requests FP16 can
fail at `TRAINER_SETUP`. A closed `TypeError` marker alone cannot establish
that this was the historical cause, and it does not authorize replay.

GRPO has a separate fixed 131-member offline inventory and 13-member outer
closure. Existing SFT inventories remain unchanged. Maintenance uses the
checked-in command, offline first because the outer closure binds its bytes:
Expand Down
38 changes: 38 additions & 0 deletions .claude/skills/fine-tuning/reference/native-local-training.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,41 @@ Resolve with the documented platform/interpreter/index and install only in a
fresh private environment with hashes required and source builds disabled.
This candidate does not select a model, enable a provider, or qualify Metal,
SFT or GRPO. Keep CUDA/Modal environments and the legacy Mac checkout unchanged.

## Prepared SFT worker candidate

`tuner/runtime/native_mlx_sft.py` is an internal CPU-tested boundary. It consumes
the existing shared `PreparedSFTExample` objects after host verification and
tokenization; it does not introduce a dataset format or public execution API.
Pass the exact prepared-input identity, tokenizer revision, retained row IDs,
declared train/validation tuples and explicit `MLXSFTConfig` controls. The
projection copies all tokens/labels immutably and records a deterministic digest.
The host must still bind those rows to the authenticated source and tokenizer.

No MLX text/chat loader, extra EOS, resplit, truncation, packing, tail dropping,
or single-offset masking is allowed. Failed masking diagnostics, already
truncated rows, attention masks other than all-ones unpadded causal rows, and
rows without shifted target supervision reject. Arbitrary ignored token spans
remain ignored. Training shuffles deterministically within its own split;
validation retains every row in declared order, including partial batches.

The custom hooks target pinned MLX-LM 0.32.2. One batch is one optimizer update;
gradient accumulation and checkpointing are deliberately unavailable in this
increment. Upstream accumulation averages microbatch means, which needs a
separate objective review for unequal supervised-token counts. Model, LoRA,
optimizer and learning-rate policy must be prepared explicitly by the future
child composition; this worker does not infer or silently apply those controls.

`tuner/runtime/native_mlx_runtime.py` compares host-supplied exact interpreter
and complete distribution pins against the running candidate before worker ML
imports. `CANDIDATE_MATCH_ONLY` means metadata and executable bytes match.
It does not authenticate wheel files, imported module origins, Homebrew's linked
Python framework, other native libraries, Metal, or launch authority. Retain the
original hash lock and use the existing `uv pip freeze`, `uv pip check`,
`shasum -a 256` and `otool -L` tools for review evidence; do not install packages
into the shared interpreter or treat a newly observed inventory as approved.

There is no native launch CLI yet. The internal worker may run only in an
independently admitted, host-owned bounded child. The desktop plan lists the
missing native runtime material, child lifecycle, artifact roles and API joins.
Do not call the worker as an ad hoc substitute for that composition.
42 changes: 42 additions & 0 deletions .claude/skills/fine-tuning/reference/sft-training.md
Original file line number Diff line number Diff line change
Expand Up @@ -460,3 +460,45 @@ The CPU `probe-filesystem` command measures actual native writer modes and host
readability under a reviewed finite container plan; it is not training, GPU,
provider or cancellation qualification. Source inventory and five-role native
verification remain closed; no default capability or UID policy is promoted.

Before short-lived source preparation, run `model-preflight` with the exact
interpreter selected for `prepare-model`; the base consumer may lack the optional
Hub SDK. This checks only the existing SDK signature contract and creates no
client or claim. `prepare-model` repeats it before its one-use claim. Use an
independently verified helper environment; do not upgrade the pinned trainer.
Create and verify necessary private source/artifact parent directories first.
Preserve any consumed model-preparation claim after failure. New failures retain
the existing allowlisted `model_preparation_stage`; never infer that stage or a
cause for an older generic diagnostic, or replay an uncertain attempt.

### Offline SFT worker rejection

The isolated source worker retains exit code 2 on failure and emits one bounded
diagnostic line to the existing `tracking/trainer.stderr.log`:
`OFFLINE_SFT_WORKER_REJECTED phase=<closed> category=<closed>`.
The phases are `BOOTSTRAP`, `CLOSURE`, `BOOTSTRAP_ORIGIN`, `ARGUMENTS`,
`IMPORT_GUARD`, `TRAINER`, and `POSTCHECK`. Categories are `REJECTION`, `IMPORT`,
`OS`, `VALUE`, `SYSTEM_EXIT`, and `OTHER`. During `TRAINER`, the last valid
existing packaged-trainer hook may append `milestone=<closed>` (for example
`TORCH_IMPORT` or `MODEL_LIBRARY_LOAD`). Unknown hooks clear that milestone.
The fallback remains the bare rejection marker. This diagnostic does not print
exception text, arguments, paths, environment values, or traceback frames;
library-generated stdout/stderr is a separate surface.

Use the labels to choose the next targeted reproduction. An import category
does not identify the missing library, and a model-load milestone does not
establish OOM. Do not interpret an older bare marker retrospectively. Preserve
the consumed claim, roots, container and retained logs; the diagnostics never
authorize replay. See `docs/review/offline-sft-worker-diagnostics.md` for the
October 20 evidence boundary, CPU regressions and fresh-run prerequisites.

After an intentional worker source change, check and refresh existing
manifests with the checked-in maintainers in dependency order:
`regenerate_offline_sft_worker_closure.py`,
`regenerate_packaged_training_worker_closure.py`,
`regenerate_grpo_worker_closures.py --kind offline`, then `--kind outer`,
`regenerate_hf_parent_closure.py`, `regenerate_modal_runtime_lock.py`, and
`regenerate_modal_inference_lock.py` (all under `scripts/`). Run each read-only
check first, review its drift, use `--write` only for intentional hash refresh,
then recheck. Preserve inventories and non-hash pins. Sync canonical skills
with `.skills/scripts/sync_skill_trees.py` and verify its `--check`.
2 changes: 2 additions & 0 deletions .skills/fine-tuning/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ Train language models with SFT, KTO, and GRPO locally or on supported cloud prov
| Prepare verified raw-text SFT data | `python tuner.py prepare-dataset --config <config.json> --json` |
| Check SFT loss masking (tokenizer only) | `python tuner.py doctor sft-mask --dataset-path <jsonl> --model <tokenizer>` |
| Inspect native training host candidate | `python tuner.py doctor native-training --json` — read-only platform/Python/package metadata; see `reference/native-local-training.md` |
| Diagnose an offline SFT worker rejection | Read the exact retained `trainer.stderr.log`; closed phase/category/milestone labels locate a boundary, not a cause or retry authority; see `reference/sft-training.md` |
| Native MLX prepared SFT candidate | Internal exact-token batch/loss and runtime comparison only; no native launch CLI or API provider yet; see `reference/native-local-training.md` |
| Plan a derived training image | `python scripts/qualify_derived_training_image.py plan --config Trainers/image_profiles/<profile>.yaml` |
| Warm Space scaffold | `python3 Trainers/cloud/scripts/manage_space.py render --template vllm_warm --output-dir /tmp/my-space --base-image ghcr.io/<org>/<image>:<tag>` |
| Warm Space deploy | `python3 Trainers/cloud/scripts/manage_space.py deploy --space-id <user>/<space> --template vllm_warm --base-image ghcr.io/<org>/<image>:<tag> --hardware a10g-small --sleep-time 3600 --var BASE_MODEL=<model>` |
Expand Down
2 changes: 1 addition & 1 deletion .skills/fine-tuning/configs/qwen35_4b_token_profile.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ model:
runtime:
provider: modal
lock_path: tuner/execution/providers/modal/modal-runtime-v1.lock.json
expected_lock_sha256: 42c25ecab7848fd5002bc27711e342d1b8fac0761621af65cba3736ad9110727
expected_lock_sha256: 4f3fa81c9aeb9130258dec23f2b841e6dbdf2d686d1cf6b50249e743c3d496ca
input:
jsonl_path: /replace/with/private/prepared-dataset.jsonl
mode: text
Expand Down
8 changes: 8 additions & 0 deletions .skills/fine-tuning/reference/grpo-training.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,14 @@ end-to-end provider deadline or public cancellation qualification. The first
executable candidate requires identical model and tokenizer revisions because
both consume one verified local snapshot.

The packaged runtime resolves `GRPOConfig` precision before loading the model.
When the retained model dtype is `null`, it passes the effective FP16 or BF16
dtype to Unsloth's loader. Pinned Unsloth checks that loaded dtype against
trainer precision during `GRPOTrainer` construction; allowing the loader to
auto-select BF16 on BF16-capable hardware while the recipe requests FP16 can
fail at `TRAINER_SETUP`. A closed `TypeError` marker alone cannot establish
that this was the historical cause, and it does not authorize replay.

GRPO has a separate fixed 131-member offline inventory and 13-member outer
closure. Existing SFT inventories remain unchanged. Maintenance uses the
checked-in command, offline first because the outer closure binds its bytes:
Expand Down
Loading
Loading