Establish representative decision quality and fix the known regression

Gap and evidence

The validation record reports 83.33% on 270 synthetic marker questions but only 42.59% on a separate 108-question known-regression set; an earlier smoke scored 50% against a retained 70% threshold. STAGE3_CORRECTNESS.md says reward training did not outperform supervised controls and Noul/Score still fail the color=red; level=1; case=99999 example. These are not representative-quality evidence.

Final native-quality acceptance is blocked by #3. Dataset/protocol preparation can proceed independently.

Acceptance

Freeze licensed, representative train/development/calibration/untouched-test sets and evaluation protocol before training, with group-disjoint splits and data hashes.
Vary option order, IDs, colors, levels, wording, domains, languages and context lengths independently; preserve the known Choice/Noul/Score failure as a regression.
Run controlled supervised, reward and calibration arms across multiple seeds and a declared compute budget; fit calibration only on calibration data and run pinned Kev/Laya baselines.
Report accuracy, NLL, Brier, ECE, ordinal error, risk/coverage, slices, uncertainty and retained failures, including the untouched test. Keep the 70% fixture threshold and avoid test-set checkpoint selection.
Demonstrate correct Choice/Noul/Score behavior on the preserved case=99999 regression or keep this issue open with the remaining measured failure.
Publish reproducible model/data/protocol revisions and distinguish measured quality from Jev comparison.

Independent protocol-preparation candidate — 2026-09-27

A seven-new-file source candidate, layev-quality-protocol.patch, was initially developed against inspected main d4de89d1990ff7cf04ee985fca42af13e2f5c79a and then revalidated using the verified source dependencies from current main e1db691cc7710ff808fa1220b16628c0d675d16c. The unchanged patch SHA-256 is c6e7e35802aa574c9418e5665a6897b7c13594c55747ee6d05a3271650b30366. It is not committed, signed, pushed, PR-published, hosted-validated or merged. No existing encoder, checkpoint, model, training entry point, API schema, telemetry or Overwatch file is changed.

The tokenizer repair was concurrently delivered by separate workstation work in PR #13. The current encoder was matched to live Git blob 00228a051ca747d55f0127aa2366f5baac588de2; data.py, io.py and schema.py were unchanged across the inspected comparison. The new protocol patch has no path overlap with that nine-file tokenizer delivery. Do not reapply the historical tokenizer patch; this candidate neither replaces it nor claims its workstation/native evidence as work performed here.

The candidate adds src/kev_laya/quality_protocol.py, a read-only scripts/verify_quality_protocol.py, two test files, documentation and deliberately incomplete protocol/data-review templates. It binds the exact protocol, data-review declaration and suite-manifest bytes; reuses the existing real data.load_suite for row/schema/hash/ID/group/normalized-state validation; rejects malformed or oversized documents, nonportable paths, symlinked inputs, changed input bytes and known smoke/counterfactual fixture IDs. It requires declarations for at least three distinct seeds, one pinned parent, matched supervised/reward step and useful-token budgets, bounded resources and zero paid use, development-only selection, calibration-only fitting, one planned untouched-test evaluation per selected arm/seed, pinned Kev/Laya baselines, required metrics/slices/variations and group-level uncertainty. The original 0.70 fixture threshold and exact case=99999 Choice/Noul/Score regression with retained failures remain mandatory in the proposed protocol.

Measured source evidence: 130 passed, zero failed/skipped on isolated Linux/Python 3.13.5, pytest 9.0.2 and Pydantic 2.13.4, both initially and after the current-main source reconciliation. These are two runs of the same suite, not 260 distinct tests. Tests use synthetic rows and explicitly artificial review declarations; they do not produce a representative benchmark or real data approval. The copied data.py, io.py, encoding.py and schema.py bytes were checked against live Git blob identities, with the encoder refreshed for PR #13. Compilation passed. Isolated new-file patch check/application/reverse-check and all seven postimage hashes passed. These are not a complete-checkout, locked, native or hosted result. Ruff was unavailable (No module named ruff); full repository checks remain required for this new patch.

The actual no-input gate invocation exits 2, reason protocol_data_review_and_suite_required, with zero training/network/paid calls and quality unmeasured. Even a structurally passing preflight explicitly leaves review authenticity and preregistration chronology unverified, representativeness unestablished, baseline/native/CUDA/context/regression outcomes untested and Jev parity unknown. Declarations and hash syntax are not approval, artifact existence, resource enforcement or proof that the protocol predates training. The known-fixture check is not an exhaustive synthetic-data detector. Externally witnessed/signed preregistration and actual run/evaluation evidence remain necessary.

Delivery artifact: jev-quality-preflight-2026-09-27.zip in the execution-session response, containing the active patch, new-file postimages, exact validation-source copies, tests and receipts. No licensed representative dataset, genuine data-review approval, native experiment or repaired regression is claimed. The primary Windows checkout/configured SSH signer and running service remain inaccessible to this session; no WSL, existing run or remote target was modified. All original acceptance boxes stay open.

Workstation delivery of preparation gate - 2026-09-27

The historical candidate-delivery status above is superseded by PR #14, now merged at f31365ea17762a88e204d63880bc02e9805c7c88. Its source commit c9688666ef0905c9cbe381390ebb8807f674e623 was SSH-signed and locally verified; GitHub verifies the merge commit. Primary local main was fast-forwarded to the same tree.

The archive manifest and external/embedded patch identity were verified. Review found one usable-input defect in the supplied candidate: an integer seed of 2**64 passed preflight but failed the trainer's torch.manual_seed. The integrated gate requires 0 <= seed < 2**64, with overflow rejection and real PyTorch endpoint tests that preserve RNG state. Seven files were added; the copied support-source files in the archive were not overlaid onto current main.

The complete Windows checkout passed 545 tests, with five explicit skips (the official TypeSafe SDK, two CUDA tests in the CPU environment, and two Windows symlink-privilege tests). The real pinned tokenizer snapshot was supplied. Indexed source integrity, compileall and critical Ruff 0.13.2 checks passed. PR CI passed all five jobs; each Ubuntu/Windows Python 3.12/3.13 suite reported 546 passed, four skipped, followed by numerical, resume/calibration/monitoring, wheel, HTTP and schema checks. Post-merge main CI also passed all five jobs on the merge commit.

The no-input CLI still exits 2, protocol_data_review_and_suite_required, with zero training/network/paid calls. No genuine reviewed representative dataset, authenticated preregistration, enforced experiment runner, native/CUDA training, pinned-baseline execution, repaired case=99999 result or Jev comparison was produced. The documented immutable-input/filesystem limits remain. All original acceptance boxes remain unchecked and this issue remains open. Native-quality execution remains dependent on #3; authentic data/protocol preparation and the controlled runner can proceed independently within their authorized scope.

Source progress for issue #5: PR #27 merged at 93bd12bd5c7e8157e861c2305b07069838f8e482 after all five required PR checks. It carries declared option-order and independent-variation strata through evaluation; missing strata remain unknown. The primary main checkout was synchronized. Postmerge run 36369727172 is still pending.

PR #28 is open at signed commit a3ca816c3676845eddd1759e446b26266c993330. It adds a durable, exclusive ledger for exact per-step useful-token reservation before model work, aggregate and per-seed/arm step/token limits, crash-pending refusal, checkpoint alignment, and an opt-in frozen-protocol CLI path. The combined local tree passed 797 tests (16 explicit skips), compilation, staged source audit and critical Ruff. Hosted checks are pending.

These are partial software gates. The representative licensed data/review, wall/GPU budget enforcement, independent variation design, multi-seed arms/baselines and uncertainty, and the retained Choice/Noul/Score regression are still outstanding. Issue #5 remains open.

Protected delivery readback: PR #27 merged at 93bd12bd5c7e8157e861c2305b07069838f8e482; all five required PR checks and all five postmerge main jobs in run 36369727172 passed. PR #28 merged at 63ddc2d576831d7da09a9a2e85e55f5bf1031d24; both source commits were verified SSH-signed, all five required PR checks and all five postmerge main jobs in run 36371367244 passed. The primary main checkout is clean and matches origin/main at the latter merge.

This completes only the declared variation-slice reporting and opt-in exact step/useful-token budget source work. Wall/GPU budget enforcement, licensed representative data and genuine review, frozen multi-seed arms/baselines, uncertainty, and the retained Choice/Noul/Score regression still require work. No representative quality measurement is claimed; issue #5 remains OPEN.

Partial source delivery for the opt-in quality resource guard: PR #30 merged at e2b60530a251ae2d29c4cca20598cf7f1946a429 after all five required PR checks passed. Local full suite: 805 passed, 16 skipped. The supervisor starts the frozen ledger wall clock before a dedicated training child, stops that child at the deadline while the supervisor lives, and records a content-free receipt and pending-step flag. No automatic retry or budget relaxation. Four of five postmerge jobs in run 36375059288 have passed; Windows 3.12 is queued.

This does not close #5. Windows descendant processes or a supervisor crash can escape this deadline, non-PyTorch GPU allocations are unmeasured, and licensed reviewed representative data, independently sampled variations, multi-seed arms, baselines, uncertainty and corrected regression results remain outstanding.

Postmerge update: all five jobs in main run 36375059288 passed. Issue #5 remains open for the measured acceptance criteria listed above.

Scoped #5 resource-enforcement repair delivered: signed PR #31 merged at 40c16be07f52a808b4f939e4c5881a0de22b82be through all five required checks. All five postmerge jobs in run 36378600335 passed. The primary checkout is synchronized to that merge commit.

On Windows, a deadline now stops the supervisor-owned process tree. A real grandchild regression proves the descendant started and cannot write after expiration. A failed tree kill raises an unverified-termination error. Local full suite: 806 passed, 16 skipped; compilation and focused Ruff passed.

This closes only the Windows descendant escape gap. Supervisor-crash durability, non-PyTorch GPU accounting, licensed data/review, native multiseed experiments, and representative quality remain open; no acceptance boxes are claimed complete.

Scoped #5 crash-containment repair: signed PR #32 merged at 8ff96d936db05792d65f90bdca92225017b3dcc3 after all five required PR checks passed. Primary main is clean and synchronized. The postmerge run has passed source and both Windows jobs; both Ubuntu jobs are still queued as of this comment.

The Windows quality supervisor now assigns its child to a kill-on-close job. A real test terminated only the supervisor after observing a started grandchild; that descendant did not complete its delayed write. Local full suite: 807 passed, 16 skipped; compile and focused Ruff passed. The spawn-to-assignment interval and POSIX supervisor crashes remain limitations, recorded in the runtime receipt. Licensed data/review, native multiseed experiments, broader resource accounting, and representative quality remain open. No #5 acceptance box is claimed complete.

Postmerge update for PR #32: all five jobs in run 36379369252 completed successfully on merge commit 8ff96d936db05792d65f90bdca92225017b3dcc3. The issue remains open for its broader acceptance criteria.

#5 preparation: signed PR #34 adds offline matched model-seed reporting. It requires at least three seed reports on the same frozen rows/targets and records per-seed quality plus observed mean, sample standard deviation and range across metrics and slices. It keeps source-group bootstrap separate and hashes the input reports.

Local source audit, compilation and critical Ruff passed; Windows full suite: 809 passed, 16 skipped. The end-to-end CLI test used local evaluation reports, not representative native multiseed runs. All five required hosted jobs are queued; PR #34 is not merged.

Licensed representative data/review, actual independent native seeds, baseline arms, retained regression repair and untouched-test reporting still remain. No acceptance box is claimed complete.

Fresh read-only native regression check on the actual Qwen LoRA step-2 checkpoint SHA-256 b6e0db32a65bbf240a5cdec8c938040054a45284e5d4cf35cd310160262b6efb: preserved color=red; level=1; case=99999 in both option orders scored 1/6 overall, Choice 1/2, Noul 0/2, Score 0/2. It chose the first option with near-unit confidence in each question. Script/JSON SHA-256 4359493f0a5a49d55125e6c99069e22753e156bf5eb84f0ae90fe0a6e4b6b852 / a5708568f145f5aeef909b652dc8bd0bc1e16e0be6d4702d64f0aac6d6d820ee. This is a two-step synthetic LoRA checkpoint, not a representative-quality or controlled counterfactual arm, so it establishes only that this checkpoint did not repair the known regression. #5 stays open.

Postmerge workflow correction for seed-report PR #34: exact merge run 36383174403 on 821ffea was cancelled by the newer PR #33 main push before all jobs completed (two Windows jobs passed; three were cancelled). All five #34 PR-head required checks passed before its protected merge, but a full exact-merge postmerge pass is not yet established. Combined main run 36384521208 is queued; the earlier exact SHA can be rerun after the newer run finishes. #5 remains open for its original quality criteria.

Independent #5 data-source research, preparation only:

Moderation workload: Google TFDS Civil Comments states a CC0 release and provides human fractional toxicity/subtype ratings plus article, parent, publication and comment IDs. Those IDs allow a proposed group-disjoint freeze. Noul thresholds, ordinal Score bins and exclusive Choice subtype rules would be derived labels requiring a reviewed, preregistered rule. The original published splits were not grouped by parent, so they cannot simply be reused for a no-leakage claim.
Question-answering workload: AI2 ARC provides natural multiple-choice science questions and reports CC BY-SA 4.0; BoolQ provides natural yes/no questions under CC BY-SA 3.0. BoolQ's published test file has no labels, so an untouched scored test needs a reviewed split from labeled records. Neither source supplies a natural ordinal Score task.
Mixed public baseline: A frozen combination could cover Choice, Noul and derived Score over more than one English short-context domain, but it would not establish general Jev replacement quality, multilingual behavior or 32k/64k context quality.
Reviewed deployment-like data: The strongest final representative gate would match the intended application, but requires an owner-approved target population, source/distribution terms, human data-use review, annotation rules and remote-transfer permission before Jev calls.

Recommended sequencing is a bounded mixed public baseline plus separately reviewed deployment-like data for the final representative claim. No dataset was downloaded, split, trained on, reviewed or approved here. The original #5 acceptance boxes remain open pending the owner's workload choice, named review, frozen protocol and controlled native runs. Public-source license statements still require a use-specific review before training or redistribution.

Public candidate acquisition update for #5, source bytes only:

ARC-Challenge was downloaded from AllenAI's publisher-linked distribution at revision 210d026faf9955653af8916fad021475a3f00453. The train/validation/test Parquet SHA-256 values are e488c1587ffdcfc8443f916c53488a95cd471c5790e0746c6bfe4cecf20962cb, 395a5c88d1580d69855fbaee9450270578df1ad5af6259771cd0a42c20e99f05, and 62f03257e737aed263f55c6abf87c7bb0028a44a6bdd2a26eb1279eb42c1d1e9.
Google BoolQ was downloaded from Google's distribution at revision 35b264d03638db9f4ce671b711558bf7ff0f80d5. The labeled train/validation Parquet SHA-256 values are 4f028e992c0bd4df30b9f056f4946b64f5c23028034ff0ed5ea467d8538cc623 and 52355d11524b4b874a9b9dcc278feb10f672d52c4f4eff9872e695ede59820f8.

Both candidates are held privately with verified file-size/hash manifests. The publisher/source pages state CC BY-SA 4.0 for ARC and CC BY-SA 3.0 for BoolQ; use-specific review remains pending. No source data is committed or redistributed. No representative dataset, group-disjoint split, derived Score labels, near-duplicate review, preregistration, training, untouched-test result, or Jev transfer was produced. This narrows the reproducibility prerequisite only; every #5 acceptance box remains open pending intended workload choice and a named data-use review.

Additional #5 candidate source acquisition, still source bytes only:

Google's Civil Comments distribution was pinned at revision f2970eb3a55777454c94069077cc8d9b5866312d. Its two train, validation and test Parquet files total 422,061,071 bytes. All four downloaded sizes and SHA-256 values match the distribution's published LFS metadata: c20f01c3aecbdd942886cacb0ee67e995df33bc712a622fde75927ed3d6ccefe, a76af69d7b24758bc38b196bc422d4767935c1dd7d3c1465554649c546d99ccb, 2e0eb65474e7e1290df8689fc93eea783160c898c09ae15382c0a9662367bd03, and 410f803ff161e7c4ba005db7de5b444cac413f4861bf056b0fd69d670d75aa08. TensorFlow Datasets states CC0 for the data and underlying comment text.

The files and provenance manifest remain private. Published splits are not established group-disjoint by article/parent/publication. No near-duplicate review, use-specific approval, Noul/Score/Choice derivation rules, Layev suite freeze, training, evaluation, Jev transfer or representative-quality claim was made. This gives the proposed public baseline a source candidate for moderation/Score, while the final #5 acceptance and intended-workload decision remain open.

Correction to the Civil Comments candidate notes: direct Parquet schema inspection of the downloaded google/civil_comments default distribution found only text, toxicity, severe_toxicity, obscene, threat, insult, identity_attack, and sexual_explicit. It does not include article, parent, publication, or comment IDs. The TFDS catalog describes richer versions/configurations, but those fields must not be attributed to these downloaded files.

Observed row counts with pyarrow 25.0.1: train 1,804,874; validation 97,320; test 97,320. The pinned file hashes remain unchanged. A source-group split by article/parent/publication cannot be built from this candidate. I updated its private provenance manifest and will evaluate a provenance-rich source before any group-disjoint freeze. No training or #5 acceptance claim occurred.

Further #5 provenance check, candidate-only and read-only:

The original Civil Comments Figshare archive, DOI 10.6084/m9.figshare.7376747.v1, reports CC0 and contains id, article_id, publication_id, and text. All five public buckets (916,484,997 bytes) were downloaded privately and checked against Figshare's published sizes/MD5; local SHA-256 values and a provenance manifest were recorded. The labeled Google Parquet export lacks those IDs, so I compared SHA-256 of exact text in both sources without exporting text or a crosswalk.

The full probe counted 1,806,317 original rows and 1,999,514 labeled rows. Of the labeled rows, 1,771,454 had one unambiguous exact source identity, 36,488 matched text with ambiguous source identities, and 191,572 had no exact source match. The labeled export repeats 2,256 distinct exact texts across its published splits, and 752 matched article IDs span those splits. Thus the published splits cannot be claimed group-disjoint; ambiguous and unmatched records require exclusion or separate review. Exact matching also does not rule out semantic near duplicates.

The private count-only receipt SHA-256 is 85c3c4715a4b7e5cb082b5bf0c195b492bb5124657d3a213e43941a0aa936925 (probe script SHA-256 e81323a854217e228c5af3308ca9dd8a01c6d08fec038260ea00d8fc06dfa0c4). This establishes a possible source-ID route for a new reviewed split, not a frozen dataset or approved training use. No records, labels, source IDs, model results, or Jev requests were published; all #5 acceptance boxes remain open.

#5 label semantics clarification: Civil Comments' toxicity and subtype values are fractions of annotators who assigned each tag, per the TensorFlow Datasets description. They can support a reviewed soft Noul target such as the probability of an annotator marking a comment toxic. Binning the fraction into Score levels would measure annotator agreement, not ordinal toxicity severity. I will not call it a natural Score quality label.

ARC has natural Choice answers and BoolQ has natural yes/no answers, but the three pinned public sources do not yet provide a natural ordered Score target. A separately reviewed Score source/annotation rule and the owner's intended workload are still needed. The private baseline-review draft records these limits and the observed group-leakage counts; no label conversion, dataset freeze, training or Jev transfer has occurred. #5 remains open.

Additional #5 natural Score candidate, source preparation only:

NVIDIA's HelpSteer2 was pinned at revision 990b2711a36180dd19d9c94b8627844866f8982a. The pinned publisher README states CC BY 4.0 and describes human 0–4 ratings for helpfulness, correctness, coherence, complexity and verbosity. Train and validation source SHA-256 values match the published LFS metadata: c0d7e91d738d42e8a08070db26c4c09a9c7631308e1f0fd380ff43d130c9f713 and 610eeb5289494d613c4c0f70aade2df8df0b499f3a24e76d232f74e6909d010a.

Read-only schema/count inspection found 20,324 train rows (10,161 distinct prompts) and 1,038 validation rows (519 distinct prompts), with all five rating fields spanning 0–4 and zero exact prompt hashes shared across the two published splits. Paired responses share prompts, so grouping by prompt plus a near-duplicate audit is required before a new train/development/calibration/untouched-test freeze. This gives the proposed public baseline a natural ordinal Score candidate for response quality, which is a separate domain from Civil Comments moderation, ARC science Choice and BoolQ yes/no QA. The owner/reviewer must still choose the intended Score attribute, wording and approved use.

Private source bytes/provenance are hash-verified; no dataset was converted, split, approved for training, trained, tested or sent to Jev. Public benchmark contamination and the absence of a separate scored HelpSteer2 test remain limitations. All original #5 acceptance boxes remain open.

HelpSteer2 source-use caveat for the preceding candidate note: NVIDIA's pinned README says the prompts are mostly user-contributed ShareGPT prompts (with a smaller Scale AI-generated portion). The dataset card's CC BY 4.0 statement alone does not complete the use-specific review of upstream prompt provenance, privacy, or remote-transfer suitability. I added this to the private candidate review draft. No HelpSteer2 rows have been converted, trained on, published or sent to Jev; #5 remains open.

#5 public-candidate split audit, count-only: a read-only NFKC/casefold/whitespace-normalized hash probe found 8 ARC-Challenge question strings crossing its published train/validation/test splits, 721 BoolQ passages crossing published train/validation, and 1 HelpSteer2 prompt crossing published train/validation. ARC source IDs did not repeat across splits, BoolQ question strings did not repeat, and HelpSteer2 had zero byte-exact prompt-hash overlap; those weaker checks would miss the observed normalized collisions. Earlier Civil Comments analysis found 2,256 exact texts and 752 matched article IDs crossing its published splits.

The private count-only receipt SHA-256 is bd0807cb9fc74c9a51fc21522499e7cb3da37260bdd919b1ad1cc8d7423cb876, script SHA-256 c04b2a51242abbf91f77b70228b996472d6c184adb5eae9c07d99c598fd9cf12. No questions, passages, prompts or answers were published. These findings support designing new reviewed group-disjoint partitions; the published splits are not a Layev frozen quality suite. Normalized equality is still weaker than semantic near-duplicate review. No data-use approval, conversion, training, test evaluation or Jev request occurred; #5 remains open.

Fresh regression readback on 2026-09-28, separate from representative quality: Layev's existing Stage 3 CPU fixture runner on source 904656cc2102ca834e765061288954a342b9511b completed 160-step supervised training, 40-step extension arms, held-out calibration and evaluation. Producer report SHA-256 1714175064e50953bd40da0efd4a5da13d5f6a9805d851b6b91db3b20d15123e; protocol SHA-256 041357cbd9cbe1024461e21df7574a65f81f9aefec6eddc87e9c9cfc6b74888d; suite manifest SHA-256 3ed3666edf3312a4dac09e95ea9ad1e3cd95d90bc4f9f0377f0c2d0513749c64. The existing fixture threshold passed, but the preserved color=red; level=1; case=99999 regression was correct for Choice and incorrect for Noul and Score (the wrong choices had probabilities about 0.99955 and 0.99999997). The report labels general predictive quality unverified and native context unverified. This run does not repair the regression, establish licensed representative data, or satisfy any original quality acceptance box. The data-source review draft remains candidate preparation pending the workload and reviewer decision.

Focused quality-readout guard delivered in PR #41, merged at 77d80e217f4e3cbc38d882c7cc532291f1707b43 with a GitHub-verified merge signature. The five required PR checks and all five post-merge main checks passed. The primary checkout is clean and synchronized to origin/main.

The CLI now durably claims each selected arm/seed before opening frozen test rows, permits one raw and one calibrated readout, and writes one paired receipt. It rejects diagnostic test access for suites outside known synthetic fixture IDs. Before scoring, it verifies the calibrated descendant has the same weights, configuration, execution policy, training steps, effective context limits, and selected training identity as the raw checkpoint. Local primary-checkout validation: 845 passed, 19 explicit environment skips, indexed source integrity, compileall, critical lint, lock check, and read-only review.

This is an incremental software guard, not representative quality evidence. There is still no approved licensed representative suite and authentic review record, multi-seed supervised/reward/calibration result, pinned-baseline result, retained-regression repair, or accepted native BF16 run. A mutable fixture ID is not an authentication boundary; unrelated code can read data directly. An invalid checkpoint detected after claim fails closed and consumes that readout without scoring. All #5 acceptance boxes remain open.

Observed-regression readout delivery is merged in PR #42 at d595e79d38c6b8d23984effe6fe254c8b36aa9cc; all five post-merge checks passed. Against the pinned actual-weight calibrated checkpoint, case=99999 still scored 1/6 across both option orders (Choice 1/2, Noul 0/2, Score 0/2). Private content-free result receipt SHA-256 b2bf3b9a53f7284756ddb03d2914881274673a38d6f6c68b5ab46e0dd5bf42eb.

The gate now preserves that failure rather than treating a fixture pass as the repair. Licensed representative data/review, frozen multi-seed experiments, baselines and actual corrected 6/6 decisions remain outstanding; this issue stays open.

The user selected reviewed deployment decisions from a real agentic workflow as #5's primary representative quality gate. The local jev-integration-evaluator skill supplies the source-review and paired-experiment method; its bundled examples and validation are synthetic, not deployment episodes or observed benefit.

The gate will first require a named real host/repository and trace source, then a source-hash-bound read-only scan and semantic review to identify actual decision producers, registered legal actions, baseline behavior, policy and downstream observations. Candidate Layev tasks are finite Choice actions, atomic Noul predicates and Score only where the host has an independently adjudicated ordered rubric. If no authentic ordinal task exists, Score needs a separately reviewed population. Preserve a valid no-useful-placement result.

No real host, reviewed decision rows, data-use/transfer review, independent labels or frozen group-disjoint split has yet been identified. Public ARC/BoolQ/Civil Comments can remain a separate development measure, not this acceptance gate. No representative quality or Jev-relative claim is made, and #5 remains open.

Navigation Menu

Repository navigation

Establish representative decision quality and fix the known regression #5

Gap and evidence

Acceptance

Independent protocol-preparation candidate — 2026-09-27

Workstation delivery of preparation gate - 2026-09-27

2 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Establish representative decision quality and fix the known regression #5

Description

Gap and evidence

Acceptance

Independent protocol-preparation candidate — 2026-09-27

Workstation delivery of preparation gate - 2026-09-27

Activity

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

2 remaining items

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

CompleteDotTech commented on Sep 28, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions