Use a verified compile-only DSPy TypeSafe client

AGENTS.md

@@ -4,6 +4,9 @@
 
 - This is a compiler/runtime library. DSPy and GEPA belong to compile time.
   Importing `typewright` or running a frozen program must not import either.
+  Native compile-time evaluation uses the qualified DSPy TypeSafe wrapper;
+  frozen execution retains the direct SDK backend. The owner approved this
+  boundary on 2026-10-03; do not move DSPy into frozen execution.
 - Target Jev's native typed API. Choice probabilities, Noul P(true), Score
   expectations, and vendor confidence are distinct values.
 - Keep the source contract fixed: output names/types, Choice labels, Score scale

README.md

@@ -48,6 +48,14 @@ flowchart LR
 DSPy and GEPA are used only at compile time. Importing `typewright` or running a
 frozen program does not import either of them.
 
+Native `compile --backend typesafe` and research selection use the experimental
+DSPy TypeSafe wrapper qualified with **DSPy 3.4.0 / SDK 0.7.0**. Install
+`.[compile]` (and `.[optimize]` for the generative teacher/GEPA). Frozen `run`,
+evaluation and held-out research execution retain the direct SDK backend and
+need only `.[live]`. This owner-approved boundary keeps DSPy out of runtime;
+offline equivalence tests do not establish live provider operation. Programmatic
+compilers can explicitly supply `DSPyTypeSafeBackend` to `ManagedBackend`.
+
 ## Where Typewright fits
 
 DSPy 3.4 added experimental, native Jev support: a `TypeSafe` client behind the
@@ -72,7 +80,7 @@ experimental upstream features that may change. Typewright's structural search a
 hierarchies are implemented but **not yet shown to improve accuracy**; the checks
 above are about process and safety, not measured gains. Planned alignment with
 DSPy: compatibility verification ([#90](https://github.com/Jev-Engineering/TypeWright/issues/90))
-and evaluating DSPy's Jev adapter at compile time ([#91](https://github.com/Jev-Engineering/TypeWright/issues/91)).
+and the qualified compile-only DSPy Jev adapter ([#91](https://github.com/Jev-Engineering/TypeWright/issues/91)).
 
 ## Quickstart
 

SETUP_PROMPT.md

@@ -98,6 +98,12 @@ the README's one-request draft smoke check with `--backend typesafe --allow-paid
 model, and usage without exposing credentials. Do not silently substitute mock
 results when live calls fail.
 
+Native compile/selection uses the experimental DSPy TypeSafe wrapper, qualified
+with DSPy 3.4.0 and SDK 0.7.0. Install `.[compile,optimize]` for live optimization;
+the generative teacher independently retains its supported 3.3.1 floor. Frozen
+runtime remains the direct SDK (`.[live]`) with no DSPy import. Offline transport
+equivalence is not authentication or live provider evidence.
+
 Live DSPy/GEPA optimization additionally needs an explicitly selected real teacher
 model and provider credentials, permission to send the projected training data to
 that provider, four disjoint splits, and agreed request/signature/token budgets.

docs/ARCHITECTURE.md

@@ -73,6 +73,21 @@ and limits are copied by the compiler, then the full graph is validated.
 `Revise` updates requested instruction/rubric entries. All use `dspy.Predict` inside an explicit
 `dspy.context(lm=...)`. A user-selected teacher provider is mandatory.
 
+The owner-approved native evaluation boundary is separate from that generative
+teacher. Compile and selection entrypoints use `DSPyTypeSafeBackend`, a lazy
+compile-only wrapper over `dspy.experimental.TypeSafe`, qualified against exactly
+DSPy 3.4.0 and SDK 0.7.0. It passes raw native question dictionaries and retains
+the complete SDK response envelope, including model, usage and Score legend;
+it disables retries, upstream caches, history, callbacks and usage tracking.
+`ManagedBackend` remains the sole pre-dispatch budget, identity, cache and leaf
+receipt owner. Unknown cost remains null. Frozen execution and held-out research
+requests use the direct `TypeSafeBackend`; importing or executing a frozen
+program never imports DSPy. Both paths have offline native payload/response and
+hierarchy reservation, settlement, resume and replay equivalence tests. Private
+experimental hooks require the exact version guard; they are not live
+qualification. Callers injecting a backend into the Python compiler retain
+ownership of that explicit choice.
+
 Standalone GEPA receives a candidate mapping `str -> str`. Each value is the JSON
 encoding of a TypeSafe entry; the entry itself may be a string, object, list, or
 null. Keys identify question instructions and indexed criteria/Score levels.

docs/INTEGRATIONS.md

@@ -7,7 +7,8 @@ Documentation inspection date: **2026-09-19**. These are the implementation targ
 | Python | >=3.11 | Core library, CLI, tests. |
 | TypeSafe SDK | `typesafe-sdk==0.7.0` | Native `TypeSafeClient.system_one`. |
 | Target model | `jev-1.13.0` | Versioned ID, not a moving alias. |
-| DSPy | `dspy[litellm]>=3.3.1,<3.5` (verified: 3.3.1 and 3.4.0) | `Signature`, `Predict`, explicit teacher `LM/context`. |
+| DSPy teacher | `dspy[litellm]>=3.3.1,<3.5` (verified: 3.3.1 and 3.4.0) | `Signature`, `Predict`, explicit teacher `LM/context`. |
+| Native compile client | `dspy[typesafe]==3.4.0`, SDK 0.7.0 (`compile` extra) | Experimental TypeSafe wrapper; absent from frozen runtime. |
 | GEPA | `gepa==0.1.4` | Standalone adapter and custom component proposer. |
 
 These are not claims that all dependencies were installed or live-tested in the
@@ -21,20 +22,22 @@ installing an arbitrary newest version.
 The supported range is `>=3.3.1,<3.5`, with `gepa==0.1.4` pinned (DSPy 3.4.0 resolves
 the same GEPA). Only 3.3.1 and 3.4.0 were exercised; later 3.4.x patch releases are
 accepted by the range but unverified. The upper bound is evidence-based: DSPy 3.4
-deprecates overriding `BaseLM.forward()` (which the teacher meter does) and schedules
-its removal in 3.5. See `BUILD_REPORT.md` for the executed results.
+deprecates overriding `BaseLM.forward()` and schedules its removal in 3.5.
+The #93 replacement uses the supported engine on 3.4; only the supported 3.3
+legacy branch still meters forward. See `BUILD_REPORT.md` for the executed results.
 
 - Engine: DSPy 3.4 `engine="auto"` prefers native lm15 execution. The native response
   has no LiteLLM `_hidden_params`, so the teacher's SDK cost estimate would silently
-  become "unknown". `DSPyTeacher` therefore passes `engine="litellm"` whenever
-  `dspy.LM` accepts `engine` (3.3.x has no such argument and always uses LiteLLM).
+  become "unknown". the #90 checkpoint therefore selected `engine="litellm"`. The #93 engine now
+  observes the raw LiteLLM response before conversion (3.3 retains its legacy path).
 - Unchanged in 3.4.0 (checked offline against a loopback test double): caches off,
   `disable_history=True` leaves no history, signature-call and LM-forward ceilings count,
   `S1_TEACHER_API_BASE`/`S1_TEACHER_API_KEY` reach the request, consent flags are untouched.
 - Pydantic: `dspy` 3.4.0 itself requires `pydantic>=2.11.0`, so the `optimize` extra
   resolves to it automatically. The base install stays `pydantic>=2.10,<3`.
-- Not covered: `dspy[typesafe]`, `TypeSafe`/`Noul`/`Choice`/`Score` decision types and
-  `ReAnchor` are not used or tested here.
+- The #90 checkpoint did not cover the native DSPy client. The #91 qualification
+  below covers raw native transport, not DSPy Predict decision-type translation
+  or ReAnchor. Those remain unused.
 - The documentation inspection date above was not changed; this verification used the
   installed packages, not a fresh read of the primary documentation.
 
@@ -121,3 +124,42 @@ the pinned `jev-1.13.0` only under the operator's own equivalence confirmation,
 not an independent attestation.
 Unexpected returned model identities fail. None of these routes has been verified
 live in this repository. See `examples/ai-gateway/README.md` for caps and setup.
+
+## Compile-only TypeSafe qualification (#91)
+
+Owner decision on 2026-10-03: use the DSPy client for native compile-time
+evaluation and retain the direct SDK for frozen runtime. Install
+`python -m pip install -e '.[compile]'`; combine with `optimize` when a generative
+teacher/GEPA is required. The teacher's 3.3.1 floor remains supported independently;
+the experimental native wrapper refuses all versions except DSPy 3.4.0 and
+typesafe-sdk 0.7.0. This deliberately fails closed on unverified patch releases.
+
+Gap analysis against actual pinned upstream code, rechecked 2026-10-03:
+
+| Upstream behavior | Qualified wrapper |
+| --- | --- |
+| Defaults to `jev-latest`; no returned-model equality check | Versioned explicit model; ManagedBackend validates exact returned identity. |
+| SDK kwargs omit RetryPolicy (SDK default is two retries) | Explicit RetryPolicy(max_retries=0), fixed timeout. |
+| Cache defaults on; history may retain raw state/questions/response | Per-client cache off; history/callbacks/tracker disabled; no upstream response retention. |
+| Response mapping drops Score legend; public result drops model/usage envelope | Raw native dictionaries and full SDK JSON envelope preserved. |
+| No paid consent or our attempt ledger | Explicit allow_paid; ManagedBackend pre-dispatch budget/cache/identity and per-leaf reservations/settlement. |
+| No billed cost evidence | Dollar cost remains null. |
+
+The wrapper uses exact-version experimental `_sdk_kwargs`, `_response` and
+`_finish` hooks. Native client traffic never goes through a generative Predict
+program. Compile CLI and flat/hierarchy research selection choose this backend;
+frozen run, evaluate and held-out study phases use the direct SDK. Callers of
+the Python compiler explicitly supply their backend. The direct SDK backend is
+retained for runtime, rather than deprecated away. The live hierarchy manifest
+binds separate compile and runtime identities; the code/manifest change needs
+a fresh digest-bound review before paid selection. It cannot revive old consent.
+
+Installed-package socket-forbidden doubles verify payload/envelope equivalence,
+real SDK constructor arguments, consent/version/model refusal, zero retries,
+cache/attempt accounting, chain/conditional/diamond/nested graph receipts,
+interrupted resume and zero-dispatch replay. These are offline mechanics, not
+live inference, authenticity, measured gains or production qualification.
+
+Primary sources: [pinned TypeSafe client](https://github.com/stanfordnlp/dspy/blob/3.4.0/dspy/clients/typesafe.py),
+[official Jev tutorial](https://dspy.ai/current/tutorials/jev_decisions/),
+[SDK usage](https://docs.typesafe.ai/sdk/python/usage).

docs/SECURITY.md

@@ -3,6 +3,16 @@
 ## Data flows
 
 A native TypeSafe request contains the projected state and all compiled questions.
+Compile/selection uses the qualified DSPy TypeSafe wrapper; frozen and held-out
+execution uses the direct SDK. The wrapper preserves native dictionaries and the
+full response identity, sets SDK retries to zero and disables upstream cache,
+history, callbacks and usage tracking. It does not route native state through a
+generative teacher. Entrypoints and provider backends enforce explicit consent;
+`ManagedBackend` owns pre-dispatch reservations, cache identity and evidence
+settlement on both paths. Experimental DSPy hooks
+are guarded to exactly 3.4.0 with SDK 0.7.0; drift fails closed. A reviewed live
+hierarchy manifest now binds separate compile and runtime backend identities,
+so an old manifest cannot approve the changed selection path by accident.
 A DSPy teacher call contains the source declaration, current plan/components, and
 selected projected training examples or training error traces with labels. The
 teacher may therefore see sensitive task data. Consent to TypeSafe is not assumed

docs/SOURCES.md

@@ -63,3 +63,12 @@ The source release intentionally uses a standalone GEPA adapter rather than
 pretending Jev provides a general text-generation API. No upstream performance
 claim is asserted for this framework. Metrics from the bundled fixture are
 synthetic software test output only.
+
+## Compile-only native client review, 2026-10-03
+
+- Exact-version upstream TypeSafe hooks: https://github.com/stanfordnlp/dspy/blob/3.4.0/dspy/clients/typesafe.py
+- Native Jev tutorial: https://dspy.ai/current/tutorials/jev_decisions/
+
+The wrapper is qualified against installed DSPy 3.4.0 and SDK 0.7.0 with
+socket-forbidden native doubles. Direct SDK frozen execution is retained by
+owner decision. No live inference, data sharing or measured gain follows.

pyproject.toml

@@ -14,10 +14,11 @@ dependencies = ["pydantic>=2.10,<3", "PyYAML>=6.0,<7"]
 
 [project.optional-dependencies]
 live = ["typesafe-sdk==0.7.0"]
+compile = ["dspy[typesafe]==3.4.0", "typesafe-sdk==0.7.0"]
 optimize = ["dspy[litellm]>=3.3.1,<3.5", "gepa==0.1.4"]
 research = ["numpy>=1.26,<3", "matplotlib>=3.8,<4"]
 dev = ["pytest>=8,<10", "pytest-cov>=5,<8", "ruff>=0.11,<1", "build>=1,<2"]
-all = ["typesafe-sdk==0.7.0", "dspy[litellm]>=3.3.1,<3.5", "gepa==0.1.4", "pytest>=8,<10", "pytest-cov>=5,<8"]
+all = ["typesafe-sdk==0.7.0", "dspy[litellm,typesafe]==3.4.0", "gepa==0.1.4", "pytest>=8,<10", "pytest-cov>=5,<8"]
 
 [project.scripts]
 typewright = "typewright.cli:main"

src/typewright/banking77.py

@@ -332,8 +332,13 @@ def plan(splits, seeds, search_calls, proposals, max_chars):
             "matching_note": "Equal budget ceilings, not equal realized tokens or proposal counts. Report actual use."}
 
 
-def make_backend(live, limit):
-    return ManagedBackend(TypeSafeBackend(allow_paid=True) if live else MockBackend(), max_calls=limit, cache=None)
+def make_backend(live, limit, *, compile_time=False):
+    if live and compile_time:
+        from .dspy_typesafe_backend import DSPyTypeSafeBackend
+        provider = DSPyTypeSafeBackend(allow_paid=True)
+    else:
+        provider = TypeSafeBackend(allow_paid=True) if live else MockBackend()
+    return ManagedBackend(provider, max_calls=limit, cache=None)
 
 
 def select_experiment(source, splits, dataset, out, *, seeds, search_calls, proposals,
@@ -539,7 +544,7 @@ def main(argv=None):
     if args.command == "select" and live and (not args.share_feedback or not args.teacher_model):
         parser.error("Live selection also requires --share-feedback and an explicit --teacher-model")
     def backend_factory(limit):
-        return make_backend(live, limit)
+        return make_backend(live, limit, compile_time=args.command == "select")
     if args.command == "select":
         def teacher_factory(limit):
             return DSPyTeacher(args.teacher_model, allow_paid=True, share_feedback=True, max_calls=limit,

src/typewright/cli.py

@@ -54,8 +54,13 @@ def make_backend(args):
         raise ConfigurationError("Choose --cache or --no-cache, not both.")
     if args.max_calls < 1 or args.request_timeout <= 0:
         raise ConfigurationError("Request budget and timeout must be positive.")
-    provider = MockBackend() if args.backend == "mock" else TypeSafeBackend(
-        allow_paid=args.allow_paid, timeout=args.request_timeout)
+    if args.backend == "mock":
+        provider = MockBackend()
+    elif getattr(args, "command", None) == "compile":
+        from .dspy_typesafe_backend import DSPyTypeSafeBackend
+        provider = DSPyTypeSafeBackend(allow_paid=args.allow_paid, timeout=args.request_timeout)
+    else:
+        provider = TypeSafeBackend(allow_paid=args.allow_paid, timeout=args.request_timeout)
     cache = None if args.no_cache else AnswerCache(args.cache)
     return ManagedBackend(provider, max_calls=args.max_calls, cache=cache)
 

src/typewright/dspy_typesafe_backend.py

@@ -0,0 +1,85 @@
+"""Optional compile-time native adapter, qualified against DSPy 3.4.0 only.
+
+Frozen programs keep using TypeSafeBackend. Importing this module does not import
+DSPy or the SDK. ManagedBackend owns all budget, cache and attempt accounting.
+"""
+from __future__ import annotations
+
+import importlib.metadata
+import os
+import re
+import time
+from typing import Any
+
+from .backends import Response, is_transient_sdk_error
+from .errors import BackendError, ConfigurationError
+from .models import Program
+
+
+class DSPyTypeSafeBackend:
+    identity = "dspy-typesafe/3.4.0+typesafe-sdk/0.7.0"
+    synthetic = False
+
+    def __init__(self, *, allow_paid: bool = False, timeout: float = 60.0):
+        if not allow_paid:
+            raise ConfigurationError("Live calls require explicit allow_paid=True / --allow-paid.")
+        if not os.getenv("TYPESAFE_API_KEY"):
+            raise ConfigurationError("Set TYPESAFE_API_KEY in your local environment; never paste it into chat.")
+        try:
+            versions = {name: importlib.metadata.version(name) for name in ("dspy", "typesafe-sdk")}
+        except importlib.metadata.PackageNotFoundError as exc:
+            raise ConfigurationError("Install DSPy 3.4.0 and typesafe-sdk 0.7.0 for compile-time evaluation.") from exc
+        if versions != {"dspy": "3.4.0", "typesafe-sdk": "0.7.0"}:
+            raise ConfigurationError("Experimental TypeSafe adapter requires dspy==3.4.0 and typesafe-sdk==0.7.0.")
+        import dspy
+        from dspy.experimental import TypeSafe
+        from typesafe_sdk import RetryPolicy
+
+        # Upstream private hooks are qualified by the exact version above. Preserve
+        # raw native dictionaries and the entire SDK JSON envelope: upstream's
+        # normal decoding drops Score legend and its public result drops identity.
+        class NativeEnvelope(TypeSafe):
+            def _sdk_kwargs(self):
+                return {**super()._sdk_kwargs(),
+                        "retry": RetryPolicy(max_retries=0, timeout=self.timeout)}
+
+            @staticmethod
+            def _response(response):
+                data = response.model_dump(mode="json")
+                if not isinstance(data, dict) or "answers" not in data or "model" not in data:
+                    raise BackendError("TypeSafe response lacks answers or model; refusing it.")
+                return data
+
+            def _finish(self, request, response, cache_hit):
+                # No DSPy history, usage tracker or shared cache. The surrounding
+                # ManagedBackend validates identity and settles native receipts.
+                if cache_hit:
+                    raise BackendError("Unexpected upstream cache hit; refusing it.")
+                return response
+
+        self._dspy = dspy
+        self._client_type = NativeEnvelope
+        self.timeout = timeout
+
+    def evaluate(self, program: Program, state: dict[str, Any]) -> Response:
+        if not re.fullmatch(r"jev-\d+\.\d+\.\d+", program.model):
+            raise ConfigurationError("Use a versioned model ID, not an alias, for measured execution.")
+        client = self._client_type(model=program.model, timeout=self.timeout, cache=False, callbacks=[])
+        start = time.perf_counter()
+        try:
+            with self._dspy.context(disable_history=True, usage_tracker=None, callbacks=[]):
+                data = client(state=state, questions={key: q.wire() for key, q in program.questions.items()})
+        except BackendError:
+            raise
+        except Exception as exc:
+            failure = BackendError(
+                f"TypeSafe request failed ({type(exc).__name__}); check local credentials, quota, and SDK compatibility."
+            )
+            failure.transient = is_transient_sdk_error(exc)
+            raise failure from exc
+        return Response(answers=data["answers"], model=data["model"], usage=data.get("usage") or {},
+                        latency_ms=(time.perf_counter() - start) * 1000)
+
+    def close(self):
+        # Upstream owns a context-managed SDK client per native attempt.
+        pass