Skip to content

refactor(downgrader): slim the converters down to what oRPC and schema libraries emit - #50

Merged
dinwwwh merged 8 commits into
mainfrom
claude/loving-mayer-07ts69
Oct 3, 2026
Merged

dinwwwh merged 8 commits into
mainfrom
claude/loving-mayer-07ts69

Conversation

@dinwwwh

@dinwwwh dinwwwh commented Oct 3, 2026 •

Copy link
Copy Markdown
Member

The downgrader exists so oRPC can generate 3.1 and 3.0 documents: it builds every document as 3.2 and runs downgradeSpecV32ToV31, then downgradeSpecV31ToV30. oRPC users also hand-write parts of that document, in base and in per-operation spec overrides, so the downgrader has to cover what people commonly write, not just what oRPC generates.

The package had grown machinery for constructs almost no one writes, which made it large and hard to follow, and slower than needed on ordinary documents. This drops that machinery. An audit then checked the result against hand-written and real-world documents, and this PR fixes the gaps it found.

main This PR
Source 1,952 lines 1,097 lines
Build 50.2 kB 30 kB
Tests 41 files, 6,149 lines 4 files, about 1,200 lines, plus a generated oRPC fixture

Engine

shared.ts goes from 1,058 to 227 lines.

  • One table-driven convertObject with a per-table memo, so a shared object is converted once and a cycle ends. clone keeps cycles in copied values the same way.
  • A per-conversion cache holds resolved $refs.
  • inline replaces a $ref by its converted target and cuts recursion with {}.
  • Any object JSON sees as one is read as one, including null-prototype and other-realm objects, such as the documents oRPC's generate() returns.
  • Removed:
    • $id/$anchor base resolution
    • exact-reuse tracing and its work budget
    • alias-chain handling
    • the fixpoint that re-ran the conversion until no reference newly dangled

Behavior changes

No longer handled. The README lists each of these under the step's limitations.

Step Edge case
3.2 → 3.1 Schema $refs into removed parts (components.mediaTypes), keeping $ids unique when inlining, pruning Links and mapping values into removed parts, dropping a parameter left without content
3.1 → 3.0 Multipart application/octet-stream and RFC6570 inference, pruning Links and mapping values into removed parts, $id-relative $refs, percent-encoded removed-part pointers, readOnly with writeOnly, deduplicating required, forcing required: true on path parameters

Kept, in a smaller form. In 3.1 → 3.0, a schema that lost a restricting keyword is marked loose. A not over it is removed, and a oneOf with a loose branch becomes anyOf, so neither rejects values the original accepts. Keywords that restrict nothing, such as zod's propertyNames: { type: 'string' }, and exactly converted tuples don't count, so a zod discriminated union keeps oneOf.

Added or fixed after the audit. The audit used hand-written documents plus 24 real 3.1/3.2 descriptions: GitHub, OpenAI, Discord, Adyen, Train Travel, Airflow, Meilisearch and others.

  • nullable: a 3.0-style nullable: true beside a single type in a 3.1/3.2 document is kept in 3.0. Before, the output rejected null.
  • Unknown keywords: keywords 3.0 doesn't define, such as zod .meta() keys or Pydantic extras, become x- extensions. Before, they made the whole 3.0 document invalid.
  • unevaluatedProperties: it becomes additionalProperties when nothing else evaluates properties.
  • Security scopes: scopes on apiKey/http requirements become [], as 3.0 requires.
  • Parameters with content: they lose style, explode and allowReserved, which oRPC's queryStyles: 'json' emits.
  • Summaries: Tag and Info summaries fill a missing description.
  • Tuples: items: false becomes maxItems. prefixItems becomes items matching any item schema, instead of items: {}.
  • Strings: a string with a contentSchema no longer becomes format: binary, and contentEncoding: binary becomes format: binary.
  • Empty enum: an empty enum becomes allOf: [{ not: {} }] instead of invalid 3.0.
  • definitions: it's read as $defs.
  • Missing targets: a $ref into a removed part whose target is missing stays as written.

Unchanged on purpose. { type: 'null' } stays { enum: [null] }, without nullable, as decided in a34a9b9 (#20).

For reviewers

  • oRPC output: compared with the published 0.1.0, it differs only in tuple items, summaries filling descriptions, key order, and the nullable that a34a9b9 (fix(downgrader): stop emitting nullable without type in 3.1 to 3.0 #20) already removed on main.
  • Cycles: an input with object cycles produces the same cycle in the output.
  • Memo trade-off: a $defs target first converted inside a recursion cut is reused with that cut wherever else it's referenced. The README lists this.

Testing

  • Tests:
    • one test file per step;
    • the official corpus, for each step and chained;
    • orpc-document.ts, generated by @orpc/openapi@2.0.0-beta.41 with Zod, validated as 3.1 and 3.0 and snapshotted;
    • a regression test for every audit and review finding.
  • Real-world documents: every collected 3.1/3.2 description whose input validates converts to a valid 3.0 document with no new dangling refs.
  • Real oRPC run: generate() for 3.2.0, 3.1.0, 3.1.1, 3.0.0, 3.0.3 and 3.0.4 validates with this build, and so does downgrading generate()'s own 3.2 output directly.
  • Benchmarks: every bench is faster than main. For example, 3.1→3.0 on the generated 100-resource API runs at about 63 ops/s against 20, and the chained official corpus at about 855 against 318. CodSpeed has the authoritative numbers.
  • Checks: pnpm test (190), pnpm lint and pnpm type:check pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PhK4SwK1yxF8aWvPBsNzVH

claude added 3 commits October 2, 2026 14:07
…a libraries emit

The downgrader exists so oRPC can generate 3.1 and 3.0 documents. It had
grown machinery for constructs almost no one writes, which made it large,
hard to follow, and slow on common documents. This rewrite keeps every
conversion that oRPC, Zod, Valibot, and ArkType output needs, and drops the
edge cases.

Engine (shared.ts, 1058 -> 199 lines):
- One table-driven `convertObject` with a per-table memo, so a shared object
  is converted once and a cycle ends, plus a per-conversion pointer cache.
- `inline` replaces a `$ref` by its converted target, cutting recursion.
- Removed: `$id`/`$anchor` base resolution, exact-reuse tracing and its work
  budget, alias chains, and the fixpoint that re-ran conversion until no
  reference newly dangled.

3.2 -> 3.1: unchanged conversions, minus `$id` uniqueness when inlining,
pruning of links and mappings into removed parts, and dropping parameters
left without content. `allowReserved` is now kept only on query parameters,
since 3.1 and 3.2 forbid it on headers.

3.1 -> 3.0:
- Kept: type arrays, const, exclusive bounds, examples, $ref siblings,
  $defs inlining, binary formats, webhooks and components.pathItems
  inlining, Reference Object stripping, mutual TLS.
- Removed: oneOf/not loosening tracking, multipart octet-stream and
  RFC6570 inference, non-OAuth scope clearing, link and mapping pruning,
  empty enum and readOnly+writeOnly handling.
- Improved: tuples become `items` matching any item schema (Zod tuples and
  maps keep their item types instead of `items: {}`), and a `$ref` into a
  removed part that dangles stays as written.

Tests: 41 edge-case files are replaced by one file per step, the official
corpus (each step and chained), and a document generated by
@orpc/openapi 2.0.0-beta.41 with Zod, validated as 3.1 and 3.0.

Benches: add the oRPC document; drop `$id` from the generated API.

The README now describes the reduced scope and its limitations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PhK4SwK1yxF8aWvPBsNzVH
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PhK4SwK1yxF8aWvPBsNzVH
@codecov

codecov Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@dinwwwh dinwwwh changed the title Refactor downgrader to simplify context and conversion logic refactor(downgrader): slim the converters down to what oRPC and schema libraries emit Oct 3, 2026
@codspeed

codspeed Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by ×2.5

⚡ 18 improved benchmarks
✅ 1 untouched benchmark
🆕 1 new benchmark

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ cyclic callback graph in a removed operation, 8 path items 6,435.4 µs 101.3 µs ×64
⚡ reference diamond into removed media types, 64 levels 3,265.6 µs 97.8 µs ×33
⚡ cyclic callback graph in removed webhooks, 8 path items 4.7 ms 1.3 ms ×3.5
⚡ property schema 138 µs 44.6 µs ×3.1
⚡ generated api, 100 resources 223.6 ms 97.3 ms ×2.3
⚡ generated api, 10 resources 20.8 ms 9.1 ms ×2.3
⚡ path item $ref chain in removed components, 200 hops 13.5 ms 6.8 ms ×2
⚡ generated api, 100 resources 385.3 ms 205.6 ms +87.38%
⚡ order schema with $defs 886.3 µs 549 µs +61.45%
⚡ generated api, 10 resources 12.2 ms 8.3 ms +46.44%
⚡ property schema 176.1 µs 124.5 µs +41.41%
⚡ generated api, 100 resources 136.7 ms 97.7 ms +39.81%
⚡ reference diamond into removed webhooks, 64 levels 4 ms 2.9 ms +38.45%
⚡ official corpus 5.9 ms 4.3 ms +36.45%
⚡ dereferenced diamond, 64 levels 943.8 µs 720.5 µs +31%
⚡ official corpus 10.9 ms 8.4 ms +30.16%
⚡ order schema with $defs 1.5 ms 1.1 ms +29.23%
⚡ official corpus 6.1 ms 5 ms +23.05%
🆕 oRPC document N/A 4.5 ms N/A

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing claude/loving-mayer-07ts69 (4dc47e2) with main (c4640c6)

Open in CodSpeed

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Two behavior regressions survive the refactor even though all 176 tests pass: a cyclic value in a copied position now throws RangeError, and dropping the loosening tracker makes not/oneOf unsatisfiable on valid input. Both are things the base converter handled.

Reviewed changes

  • Minimal Context — root/targets/seen/inlining replace the location, resource, inlining, copy-tracking, and exact-reuse state; the fixpoint downgrade driver and its dangles/alias machinery are gone.
  • Simpler core — clone lost its memo, convertObject caches by (fields map, source object) and returns the in-progress object for cycles, Finish mutates in place and returns void, map/list reworked.
  • Converters rebuilt — $id/anchor/external ref resolution and loosening tracking removed; $defs and removed parts are inlined by local JSON Pointer; Path Item $ref merging, tuple rewriting, and multi-type handling rewritten in v3.1→3.0.
  • Tests replaced — granular spec/schema suites deleted and consolidated into snapshot-backed v3.1-to-v3.0.test.ts / v3.2-to-v3.1.test.ts, plus a new oRPC integration test.

⚠️ This changes behavior, not just structure

The PR body frames this as a refactor that "maintains backward compatibility of the public API", but the resource/$id resolution and the loosening tracker are removed, so documents the base converter turned into safe (looser) schemas now yield unsatisfiable schemas or throw on cycles — the two inline findings below. The README intro still leads with "loses detail, never meaning" while its new Limitations section says a not/oneOf "can then reject values the original accepts"; the deleted granular tests were the only thing pinning the old guarantee. Please confirm the guarantee is intentionally dropped for published consumers (oRPC) and reword the intro, rather than presenting this as behavior-preserving.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

Comment thread packages/downgrader/src/shared.ts Outdated
Comment thread packages/downgrader/src/v3.1-to-v3.0.ts
…the stack

`clone` recursed without a memo, so a cycle reached through a copied
position, such as `example`, `default`, or an extension, threw a
RangeError. It now copies each object once and keeps the cycle, as
converted positions already did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PhK4SwK1yxF8aWvPBsNzVH

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • clone keeps cycles in copied values. clone now delegates to a memoizing copy, which records each container's output before recursing, so a cyclic value reached through a copied position — an unlisted keyword, default, example, an x- extension, or a non-converted map entry — is reproduced as a cycle instead of overflowing the stack.
  • Regression tests added. One per converter, asserting a self-referential example/default round-trips as a cycle while converted and copied positions both stay within their own graph. Both fail on the previous commit and pass now.

pnpm test (178 pass) and pnpm lint are clean. This is submitted as a comment rather than an approval only because the earlier not/oneOf loosening thread is still open; the new commit does not touch that code, so it introduces no new concerns.

Pullfrog  | Fix it ➔ | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

…word is removed in 3.1 to 3.0

Removing a restricting keyword, such as `contains` or `propertyNames`,
loosens a schema. A `not` over it then rejected values the original
accepted, as `{ not: { contains: … } }` became `{ not: {} }`, and a
`oneOf` whose branches now overlap rejected values matching both.

A converted schema is now marked loose when it lost such a keyword, holds
a loose subschema, or is a `$defs` target cut at recursion. A `not` over
it is removed and a `oneOf` with a loose branch becomes `anyOf`. Only a
loose schema reached through a `$ref` kept in the output stays a
limitation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PhK4SwK1yxF8aWvPBsNzVH

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • Restored not/oneOf loosening tracking. v3.1-to-v3.0.ts now defines RESTRICTING_KEYWORDS and a module-level LOOSE WeakSet; a converted schema is marked loose when it lost a restricting keyword, holds a loose subschema, or is a $defs target cut at recursion, and finishSchema deletes an enclosing not and lowers a oneOf with a loose branch to anyOf.
  • Updated the README. Added not/oneOf rows to the removal table and narrowed the step's limitation to the $ref-kept case.
  • Added regression tests. Cover not removal (direct, through properties, and through the recursion cut), the oneOf→anyOf rewrite, and the case that stays not.

This addresses the prior review's loosening finding: the new tracker is smaller than main's and its only remaining gap is the documented kept-$ref case. pnpm test (181), pnpm lint, and pnpm type:check pass; snapshots are unchanged.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

oRPC users hand-write parts of the 3.2 document too, in `base` and in
operation `spec` overrides. An audit with hand-written documents and 24
real 3.1/3.2 descriptions (GitHub, OpenAI, Discord, Adyen, Train Travel,
and others) found these gaps:

- Objects with a null-prototype class or from another realm, such as the
  document oRPC's generate() returns, came back unconverted.
- 3.1 -> 3.0 dropped a 3.0-style `nullable: true` beside a type, so the
  3.0 output rejected null. It is kept now, and counts as loosening.
- Scopes on apiKey and http requirements, which 3.0 forbids, are `[]`.
- Keywords 3.0 does not define, such as zod `.meta()` keys, made the
  whole 3.0 document invalid. They become `x-` extensions.
- `unevaluatedProperties` with nothing else evaluating properties becomes
  `additionalProperties` instead of being removed.
- `propertyNames: { type: 'string' }` and exactly converted tuples no
  longer count as loosening, so a zod discriminated union keeps `oneOf`.
  `items: false` becomes `maxItems`.
- A parameter with `content` loses `style`, `explode`, and
  `allowReserved`, as oRPC's `queryStyles: 'json'` emits.
- Tag and Info summaries fill a missing description.
- A string with `contentSchema` no longer becomes `format: binary`.
- An empty `enum` becomes `allOf: [{ not: {} }]` instead of invalid 3.0.

The README lists each rule, and new limitations for external `$ref`s,
mTLS-only operations, shared inlined targets, and TypeBox records.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PhK4SwK1yxF8aWvPBsNzVH

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The bdc2da3 loosening refinement is sound on its own, but the refactor dropped the guards (and the tests) that kept a contradictory const / type: "null" schema from being treated as exact, so an enclosing not / oneOf now rejects values the 3.1 source accepted. main handled both cases and pinned them in the deleted tests/v3.1-to-v3.0/schema/loosening.test.ts; the consolidated suite does not cover them.

Reviewed changes

  • 3.0-only schema keywords — unknown keywords are renamed to x- extensions, an empty enum becomes allOf: [{ not: {} }], and a 3.0-style nullable is kept beside a single type (and marks the schema loose).
  • Sharper loosening detection — losesRestriction no longer counts propertyNames: { type: "string" }, an exactly-converted tuple, or unevaluatedProperties already expressed as additionalProperties; convertTuple returns whether the conversion is exact and maps items: false to maxItems.
  • unevaluatedProperties → additionalProperties — converted when no in-place applicator evaluates properties.
  • Document-level niceties — a 3.1 info.summary and a 3.2 tag.summary fill a missing description; a parameter with content also drops style, explode, and allowReserved; scopes on apiKey / http security requirements are emptied.
  • JSON-like inputs — isRecord accepts null-prototype and other-realm objects (oRPC documents) and excludes arrays.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

Comment thread packages/downgrader/src/v3.1-to-v3.0.ts
claude added 2 commits October 3, 2026 04:46
An adversarial review of bdc2da3 confirmed these:

- A tuple's item schemas were deduplicated by their converted output, so
  an item that lost a restriction could hide behind an equal one, and the
  tuple counted as exact. An enclosing `not` or `oneOf` then rejected
  values the original accepts. Items are now deduplicated by their source
  schema, and a tuple with a loosened item is never exact.
- An existing `x-` extension holding null was overwritten by the renamed
  keyword.
- A `const` outside the `enum` beside it, and `type: 'null'` with an
  `enum` that lacks null, matched nothing; their 3.0 form matched values
  without counting as loosened. The first now counts, and the second
  keeps matching nothing.
- `isRecord` accepted objects that inherit keys from a null-prototype
  parent and read those keys, which JSON ignores.
- `definitions`, renamed to `x-definitions`, left `$ref`s into it
  dangling. It is now read as `$defs`, as 2020-12 still does.

The README now says external content `$ref`s in 3.2 are removed, and
that a cycle of objects can hide a loosened schema from `not` or `oneOf`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PhK4SwK1yxF8aWvPBsNzVH
Valid schemas that combine keywords unusually, own __proto__ keys, a
cyclic tuple item, non-local or looping $refs, and malformed values the
converters must pass through without throwing. Coverage of src is now
100% of lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PhK4SwK1yxF8aWvPBsNzVH

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

This run reviewed e1a1eb6, the fix commit that resolves the prior review's inline findings.

  • Deduplicated tuple item schemas by source — convertTuple keys variants by the item's source schema (jsonKey(item)) and returns the tuple as inexact when the retained item is loose, so a loosened item can no longer hide behind an equal one.
  • Preserved an x- extension holding null — the unknown-keyword renamer uses Object.hasOwn in place of ??=, so an existing x- key valued null is not overwritten.
  • Closed the const / type: "null" inversions — a const outside the enum beside it now marks the schema loose, and type: "null" with an enum that lacks null stays unsatisfiable via allOf: [{ not: {} }] (an enum containing null becomes [null] exactly).
  • Tightened isRecord for inherited keys — a null-prototype parent with own keys is no longer read as a JSON object, so keys an object would inherit are ignored as JSON ignores them.
  • Read definitions as $defs — added to the removed-keyword set and the inline check, so $refs into it are inlined instead of left dangling; README updated.

Verified: pnpm test (190 pass), pnpm lint, and pnpm type:check are clean. A differential Ajv 2020-12 fuzz over the new const/enum/type: "null"/prefixItems paths (269 schemas × 16 instances, ~3.8k checks) found no value the 3.1 source accepts that the 3.0 output rejects.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

This run covered 4dc47e2, a test-only commit layered on the e1a1eb6 fixes.

  • Added unusual-combination coverage — $ref with siblings plus a type array and anyOf, type: "null" with allOf, empty enum with allOf, a multi-type array, a single-element type array, and a maximum/exclusiveMaximum conflict, each pinned to an exact output.
  • Added own-__proto__ coverage — __proto__ keys in properties and in a copied example survive as data keys without touching the prototype.
  • Added a cyclic tuple-item test — two prefixItems entries sharing one cyclic schema deduplicate and the cycle is reproduced in the output.
  • Added $ref-resolution boundaries — external and percent-invalid refs stay as written, and securitySchemes self-refs terminate.
  • Added malformed pass-through coverage — non-array security/parameters/responses and non-record properties/allOf are cloned through without throwing.

Every added assertion is an exact .toEqual or identity check that would fail if the pinned behavior regressed, so none is theatre. pnpm test (197 pass), pnpm lint, and pnpm type:check are clean.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

@dinwwwh
dinwwwh merged commit 3b11d1f into main Oct 3, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants