Skip to content

Add plugin-hook-based scoped PostgreSQL introspection - #3

Draft
Zetazzz wants to merge 4 commits into
mainfrom
agent/scoped-introspection-plugin-hook
Draft

Add plugin-hook-based scoped PostgreSQL introspection#3
Zetazzz wants to merge 4 commits into
mainfrom
agent/scoped-introspection-plugin-hook

Conversation

@Zetazzz

@Zetazzz Zetazzz commented Aug 21, 2026

Copy link
Copy Markdown

What this changes

PostgreSQL introspection currently reads the full catalog even when a service
exposes only a small set of schemas. On databases with large catalogs, most of
that work and result data is unrelated to the generated GraphQL schema.

This PR adds opt-in schema-scoped introspection. It starts from the schemas in
pgService.schemas, recursively retains the catalog objects needed by those
schemas, and validates the result against the same plan that built the query.

When scoped introspection is omitted or disabled, the service uses the existing
stock query. The stock SQL and the existing parse, cache, watch, registry, and
entity-event lifecycles remain unchanged.

This PR and #2 use the same pg-introspection query-plan API and scoped
algorithm. This PR keeps scoped policy in an optional plugin and integrates it
through a neutral query hook; #2 is the direct-integration alternative. Only
one of the two PRs should merge.

Configuration

The scoped plugin is not part of defaultPreset. Install the dedicated
preset, then configure services using the name of an existing entry in
pgServices.

The preset below is abbreviated: mainPgService and
analyticsPgService represent the existing connection, adaptor, and other
service settings.

import { PgScopedIntrospectionPreset } from "graphile-build-pg";

const preset: GraphileConfig.Preset = {
  extends: [PgScopedIntrospectionPreset],

  pgServices: [
    {
      ...mainPgService,
      name: "main",
      schemas: ["app_public"],
    },
    {
      ...analyticsPgService,
      name: "analytics",
      schemas: ["analytics"],
    },
  ],

  gather: {
    pgScopedIntrospection: {
      // Matches pgServices[0].name.
      main: true,

      // Matches pgServices[1].name.
      analytics: {
        catalogTypes: "dependency-closure",
        capabilityExtensions: ["pg_trgm"],
      },
    },
  },
};

The keys main and analytics are not fixed names or schema names. Each
key must match a pgServices[].name. The matching service's schemas
array supplies the scoped root schemas, so schemas are not repeated inside
pgScopedIntrospection.

Setting pgScopedIntrospection does not install the plugin. If neither
PgScopedIntrospectionPreset nor PgScopedIntrospectionPlugin is
installed, the configuration is not consumed and the service continues to
use stock introspection.

Service value Behaviour
omitted Use stock introspection
false Use stock introspection
true Use scoped introspection with defaults
object Use scoped introspection with the supplied options

Services omitted from pgScopedIntrospection continue to use stock
introspection.

Options

interface SchemaScopedIntrospectionOptions {
  catalogTypes?: "all" | "dependency-closure";
  capabilityExtensions?: readonly string[];
}
  • catalogTypes: "all" is the default. It retains types required by the
    dependency closure plus all types in pg_catalog. This is the more
    conservative compatibility setting.
  • catalogTypes: "dependency-closure" retains only types reachable from the
    scoped roots and their dependencies.
  • capabilityExtensions explicitly retains extension metadata even when no
    retained object currently depends on that extension. It retains the
    extension record, not every object owned by the extension.

Extensions required by retained tables, indexes, functions, or support objects
are discovered automatically. They do not need to be repeated in
capabilityExtensions.

Schema and dependency behaviour

  • root schemas come only from pgService.schemas
  • referenced relations, constraints, function signatures, domains, arrays,
    ranges, multiranges, indexes, inheritance parents, and extension metadata
    enter the dependency closure automatically
  • dependencies may cross schema boundaries when a retained object requires
    them
  • there is no dependency-schema allowlist
  • unrelated schemas and catalog objects are not retained merely because they
    have similar names or belong to the same extension

Fail-fast behaviour

With the scoped plugin installed:

  • configuring an unknown PostgreSQL service name fails
  • enabling scoped introspection for a service without root schemas fails
  • a missing configured root schema fails after the result is parsed
  • missing dependency types and dangling type references fail validation

Ownership and execution flow

Concern Owner
Scoped SQL, CTEs, and parameters pg-introspection
Normalized scope and query plan pg-introspection
Result integrity validation pg-introspection
Service configuration and plugin policy PgScopedIntrospectionPlugin
Query execution and lifecycle PgIntrospectionPlugin

There is no additional src/scopedIntrospection.ts adapter. The independent
PgScopedIntrospectionPlugin.ts is the thin adapter required by this design:

PgIntrospectionPlugin creates the stock query event
  -> query hooks run in resolved plugin order
  -> PgScopedIntrospectionPlugin may replace the query with plan.query
  -> PgIntrospectionPlugin executes and parses the final query
  -> PgScopedIntrospectionPlugin validates results belonging to its plan
  -> PgIntrospectionPlugin emits the existing entity events

The stock plugin does not read scoped configuration or know scoped policy. The
trade-off, and the main review question for this PR, is the new reusable query
replacement contract.

Hook contract

  • hooks run in resolved plugin order
  • each hook sees the query selected by the previous hook
  • when multiple hooks replace the query, the last replacement wins
  • scoped validation follows the exact query object produced by the scoped plan;
    a later replacement is not incorrectly validated as scoped
  • configuration without the scoped plugin leaves the stock query unchanged

Compatibility and safety

  • makeIntrospectionQuery() retains its existing public behaviour and exact
    generated SQL; a golden test protects it
  • scoped introspection and its plugin are opt-in
  • defaultPreset is unchanged
  • dependency schemas are discovered rather than configured
  • validation uses the normalized scope carried by the executed query plan
  • no lockfile, parse lifecycle, cache, watch, or entity-event behaviour changes

Validation

  • pg-introspection: build passed; 46/46 tests passed
  • owner tests cover plan normalization, missing roots, missing function types,
    dangling column types, and types retained in internal lookups
  • graphile-build-pg: build passed; 28/28 tests passed
  • hook tests cover opt-in installation, plugin order, repeated replacement,
    last-writer-wins, and configuration without the plugin
  • real PostgreSQL integration verifies stock/scoped schema equivalence, runtime
    execution, cross-schema types and functions, ranges and multiranges, foreign
    keys, inheritance, ordinary indexes, and pg_trgm GIN/GiST indexes
  • affected ESLint and Prettier checks passed
  • git diff --check passed; no format-only changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant