Skip to content

feat(lp): add learning progressions generation, tests, docs, and evaluation - #32

Closed
tonyzhao6 wants to merge 100 commits into
mainfrom
tz6/lp-kg-build-step-29
Closed

tonyzhao6 wants to merge 100 commits into
mainfrom
tz6/lp-kg-build-step-29

Conversation

@tonyzhao6

Copy link
Copy Markdown
Contributor

Reviewer: @MustafaAkolawala


Main Changes

  1. Implemented LP KG artifact generation after AS + LC steps.
  2. Added docs for LP KG pipeline
  3. Added tests for LP KG modules

To-do before merge (optional)

Best way to review PR is to start with tz6/lp-kg-build-step-2 branch and go thru one build step at a time to understand each new file change/addition. Can also skip branches if you want to review multiple files at a time. The engineering brief in artifacts/instructions is good context for understanding the overall goal of this PR and the build steps.

Checklist

Fill with x for completed.

  • My code follows the style guidelines of this project
  • I have reviewed my own code to ensure good quality
  • I have tested the functionality of my code to ensure it works as intended
  • I have resolved merge conflicts
  • I have ensured that my PR branch is up to date with the merge branch

(Delete any items below that are not relevant)

  • I have updated the test suite
  • I have updated dependencies
  • I have updated affected documentation (including READMEs)
  • I have updated scripts in .github/
  • I have updated CDK stacks

tonyzhao6 added 30 commits August 31, 2026 16:18
- record approved D1-D14 decisions and accepted LIMITs
- align role guidance and release gates with D12-D14
- define the approved Steps 24-29 workflow
tonyzhao6 added 28 commits September 15, 2026 16:16
- Isolate evidence construction from production nominations and judgments
- Add reconstructed, expanded, LC-removed and hierarchy-removed conditions
- Preserve warnings and record collateral metadata omissions
- Bind immutable payloads to source hashes, references and construction audits
- Index framework-contained pairs with bounded memory and exact counts
- Implement ordered upstream and production diagnostic tags
- Add Unicode normalization and exact token Jaccard calculations
- Reconcile production outcomes, publications and correction provenance
- Bind population identities to frozen input material
- sample production outcomes with deficit-only diagnostic supplements
- independently sample upstream uniform and diagnostic cohorts
- retain selection routes, exact probabilities, and quota shortfalls
- freeze material-bound sample plans and common upstream evidence
- retain bounded nomination facts while redacting recommendations
- preserve original production evidence and operative rationales
- record field-level mappings and distinct material identities
- retain producer drafts for correction diagnostics
- define separate classification and rationale-grounding schemas
- render evidence-only prompts with exact material hashes
- construct five control families with hidden expectations
- preserve citation boundaries and opaque request identities
- freeze all conditions, replicates, prompts and dependencies
- add presentation and evidence-removal variants
- map displayed judgments to canonical endpoint directions
- bind schedules to frozen inputs, implementation and judge settings
- freeze complete invocation schedules in content-addressed storage
- record attempts and usage transactionally under an exclusive writer lock
- validate cached responses against exact scheduled identities and evidence
- resume frozen selections without rediscovering curriculum runs
- preserve judgment caches across scorer-only changes
Retain the generation lock after successful production completion.
Enforce grounding consistency while preserving valid ambiguity.
Align evaluator configuration tests with the approved model default.
Add offline grounding, retry, cache, and model-availability coverage.
…checks

Preserve the fixed development selection and original artifact hashes while
capturing current lock state in temporary snapshots. Add regression coverage
rejecting lock appearance after freezing.

Validate the full offline suite including slow tests: 4167 passed.
Use synthetic temporary snapshots and a fixed synthetic historical fixture.
Preserve full validation and material binding, reject repository-results
reads, and cover corruption across both checkpoint formats.

Validate the full offline suite including slow tests: 4179 passed.
…P responses

Bind effective output schemas, modes and SDK versions.
Validate original provider envelopes before SDK normalization.
Preserve safe diagnostics, raw output and per-attempt usage.
Cover bounded retries, durable exhaustion and cache compatibility.
Expose canonical citations for nonblank historical producer and checker
policies while preserving strict validation and separate grounding.

Add synthetic coverage for citation boundaries, blind classification,
material identity changes, and rejection without ledger mutation.
Correct the request-only citation oracle.
Document exact historical-policy citations and stale evaluator invocation
rejection while preserving prior judgments and attempt history.

Distinguish stale production reuse from authorized regeneration, retaining
unsupported-checkpoint rejection before any overwrite effects.
Cover nested Boolean/number differences, exact values, absent/null
distinctions, pointer escaping, and independent message equality.
Verify payload and evidence-family reporting in both directions.

Validation: 52 presentation tests pass. Also updated docs.
Support all six projection/checkpoint combinations with strict material binding.
Reject exhausted historical failures recovered in the same run.
Preserve valid retry recovery, immutable evidence and schedule-qualified caches.
Reject saved-stage regression, backward request progress, and
later-request attempts before earlier completion.

Add historical chronology regressions and validate compatibility
lifecycles and affected production safeguards.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant