feat: filters become an AND/OR tree (tool panel SP2a)

.changeset/filter-tree-core.md

@@ -0,0 +1,44 @@
+---
+"@pretable/core": minor
+---
+
+Filters are an AND/OR tree.
+
+`PretableQueryFor.filters` is still an array, and an array of plain leaves
+still means exactly what it meant before — the top level is an implicit AND.
+What is new is that an element may also be a **group**:
+
+```ts
+interface PretableFilterGroupFor<TColumns> {
+  readonly op: "and" | "or";
+  readonly children: readonly PretableFilterNodeFor<TColumns>[];
+}
+```
+
+Groups nest, so a query can express any AND/OR shape. `PretableFilterNodeFor`
+is the union of a typed leaf and a group — the type most call sites reading
+`filters` want — and `isPretableFilterGroup(node)` narrows one to a group. The
+guard checks the group's own fields positively, so an unrecognized shape fails
+closed rather than being treated as a branch with no children.
+
+Two rules a consumer has to know:
+
+- **An EMPTY group holds — for BOTH operators.** `{ op: "or", children: [] }`
+  keeps every row, exactly like `{ op: "and", children: [] }`. Naive algebra
+  says an empty OR is false; that answer is wrong for a product, because a
+  group the user is still assembling in a filter builder would blank the grid
+  the moment it appeared. An empty group constrains nothing.
+- **Nesting is bounded at 64 levels below the root.** Top-level elements sit at
+  depth 0, so a node at depth 65 — a group nested more than 64 deep — makes
+  `compileQuery` fail the query with `code: "invalid-query"` and a
+  `query.filters[i].children[j]…` path. This is a new reason for an existing
+  rejection, and the only way an otherwise well-formed query can now be
+  refused.
+
+Evaluation, query equality (so plan reuse and recompile decisions), capture and
+freezing, and `distinctValues` all recurse. Equality stays order-insensitive
+per level, which AND and OR both license.
+
+Nothing in this release builds a group on its own — no UI renders or authors
+one yet. `@pretable/react` ships the surface half alongside: funnels light on a
+filter at any depth, and the per-column menu owns only its top-level leaf.

.changeset/filter-tree-surface.md

@@ -0,0 +1,39 @@
+---
+"@pretable/react": minor
+---
+
+The surface speaks filter trees: funnels, the column menu, and controlled
+state.
+
+`query.filters` is now an AND/OR tree — each element is either a typed leaf or
+a `{ op, children }` group, and groups nest (see `@pretable/core` for the node
+type, the `isPretableFilterGroup` guard, and the empty-group rule). The
+surface's chrome follows:
+
+- The **funnel** lights on ANY occurrence of a column, at any depth. A filter
+  the user built inside a group still removes their rows, so it still shows as
+  a filter on that column. Previously the surface kept a per-column record
+  projected out of the query; a group carries no `columnId`, so that record
+  would have collapsed every group onto the single key `undefined` and left the
+  funnel dark. The record is gone — the surface holds the tree verbatim.
+- The **column filter menu** owns exactly its column's FIRST top-level leaf. It
+  hydrates from that leaf (never from one nested in a group), and a commit
+  replaces it in its existing slot rather than removing it and appending at the
+  end. Every group element passes through by reference: a menu commit cannot
+  edit, reorder, or drop a branch it did not author, and clearing a column
+  removes only its top-level leaf. Two ordering details change for a
+  hand-authored `filters` that carries duplicate top-level leaves for one
+  column — nothing the menu can produce: the menu now reads the FIRST of them
+  (the per-column record it replaced was last-wins), and a commit collapses
+  them to the single leaf it just wrote.
+- **Controlled queries** take the tree shape. A controlled `query.filters`
+  containing groups renders funnels and filters rows exactly as the engine
+  evaluates it.
+- `LabeledGridSurface`'s `is-filtered` header decoration walks the tree by the
+  same "occurrence anywhere" rule.
+
+`isPretableFilterGroup`, `PretableFilterGroupFor` and `PretableFilterNodeFor`
+are re-exported from `@pretable/react` — a consumer reading `onQueryChange`'s
+`filters` needs the guard to tell leaves from groups.
+
+No UI builds groups yet; nothing in this release deepens a tree on its own.

apps/website/app/api/docs/rows/__tests__/dataset.test.ts

@@ -457,6 +457,55 @@ describe("queries this fixture cannot answer", () => {
       /array of selected values/,
     );
   });
+
+  /*
+   * On the wire `query.filters` is an AND/OR tree, and this fixture answers
+   * leaves only. Rejecting is the posture the server-data overview documents,
+   * so it is pinned here — including in the shapes where the rejection was
+   * NOT reached before the check moved ahead of the row loop. Each of these
+   * three returned 200 with zero rows, which is a result computed from less
+   * than the query asked for and reads to a reader as "nothing matched".
+   */
+  const GROUP = {
+    op: "or",
+    children: [],
+  } as unknown as DocsQuery["filters"][number];
+
+  test("a filter group is rejected by name, not by a message about a column", () => {
+    expect(() =>
+      applyDocsQuery(DOCS_ORDERS, { ...EMPTY_DOCS_QUERY, filters: [GROUP] }),
+    ).toThrow(/carried a filter group at query\.filters\[0\]/);
+  });
+
+  test("a filter group behind a leaf that matches nothing is still rejected", () => {
+    expect(() =>
+      applyDocsQuery(DOCS_ORDERS, {
+        ...EMPTY_DOCS_QUERY,
+        filters: [
+          { columnId: "region", operator: "isAnyOf", value: ["Nowhere"] },
+          GROUP,
+        ],
+      }),
+    ).toThrow(/carried a filter group at query\.filters\[1\]/);
+  });
+
+  test("a filter group over no rows at all is still rejected", () => {
+    expect(() =>
+      applyDocsQuery([], { ...EMPTY_DOCS_QUERY, filters: [GROUP] }),
+    ).toThrow(DocsQueryError);
+  });
+
+  test("but a leaf-only query over the same shapes still answers", () => {
+    expect(
+      applyDocsQuery(DOCS_ORDERS, {
+        ...EMPTY_DOCS_QUERY,
+        filters: [
+          { columnId: "region", operator: "isAnyOf", value: ["North"] },
+        ],
+      }).length,
+    ).toBeGreaterThan(0);
+    expect(applyDocsQuery([], EMPTY_DOCS_QUERY)).toEqual([]);
+  });
 });
 
 describe("totalFor", () => {

apps/website/app/api/docs/rows/dataset.ts

@@ -9,6 +9,39 @@ export interface DocsOrder {
 }
 
 export interface DocsQuery {
+  /**
+   * LEAF-ONLY, AND KNOWINGLY BEHIND THE ENGINE. `PretableQueryFor.filters` is
+   * an AND/OR TREE: an element is either a typed leaf or a
+   * `{ op, children }` GROUP, nestable. This shape admits leaves only.
+   *
+   * Nothing catches the mismatch at compile time, and that is not an
+   * oversight to be fixed by a cast: the type boundary is genuinely severed
+   * by `JSON.stringify` in each example's `fetch-rows.ts` — a query leaves the
+   * client as text and arrives here as `unknown`, so `pnpm typecheck` is green
+   * over a real gap.
+   *
+   * So the rejection is a RUNTIME one, and it is deliberate rather than
+   * incidental: `applyDocsQuery` scans `filters` for `children` BEFORE it
+   * reads a row and throws `DocsQueryError` naming the group, and the route
+   * answers with an error rather than with wrongly-filtered rows.
+   *
+   * Before the scan, per-row was the only check, and it was reachable only
+   * when a row survived the leaves ahead of it — so a leaf matching nothing,
+   * or an empty dataset, answered 200 with zero rows and no throw at all. See
+   * `applyDocsQuery` for why well-formedness is asked once, of the query.
+   * (Left to itself the mismatch also failed — a group has no `columnId`, so
+   * `columnTypeFor` threw — but about a column, which is not what went
+   * wrong.)
+   *
+   * Nothing in the docs builds a group yet — the built-in column menu writes
+   * top-level leaves only — so no example can reach this today. A server
+   * meeting a real tree has three honest choices (reject, flatten when every
+   * join is AND, or implement the recursion). This fixture REJECTS, by name,
+   * in `matches()`: it is a demo of the wire contract, not a filter engine,
+   * and implementing the recursion here would teach nothing the engine does
+   * not already do. The contract itself is stated on the section overview,
+   * `content/docs/server-data/index.mdx`.
+   */
   filters: readonly {
     columnId: string;
     operator: string;
@@ -429,10 +462,36 @@ function matchesText(
   }
 }
 
+/**
+ * One wording for the one thing this fixture refuses, so the up-front scan in
+ * `applyDocsQuery` and the per-row branch in `matches()` cannot drift apart.
+ */
+function rejectFilterGroup(index?: number): never {
+  const where = index === undefined ? "" : ` at query.filters[${index}]`;
+  throw new DocsQueryError(
+    `This fixture answers leaf filters only, and this query carried a filter group${where}. ` +
+      "A server that does not implement AND/OR groups must say so rather " +
+      "than drop them: see /docs/server-data.",
+  );
+}
+
 function matches(
   row: DocsOrder,
   filter: DocsQuery["filters"][number],
 ): boolean {
+  /*
+   * The rejection this fixture owes the wire contract, said out loud. On the
+   * wire `query.filters` is an AND/OR tree (see `DocsQuery` above), and a
+   * group carries `children` where a leaf carries `columnId`.
+   *
+   * Without this branch a group was already rejected — `columnTypeFor`
+   * throws on the missing `columnId` — but with `Unknown column
+   * "undefined"`, a message about the wrong thing entirely. A fixture whose
+   * job is to teach that the server applied the filter has to name the
+   * reason it did not.
+   */
+  if ("children" in filter) rejectFilterGroup();
+
   const type = columnTypeFor(filter.columnId);
   assertUsable(filter.columnId, type, filter.operator, filter.value);
 
@@ -459,6 +518,24 @@ export function applyDocsQuery(
   rows: readonly DocsOrder[],
   query: DocsQuery,
 ): DocsOrder[] {
+  /*
+   * The group rejection has to happen HERE, before a single row is read.
+   * `matches()` carries the same test, but it runs per row inside the loop
+   * below, so it is reachable only if some row survives every earlier leaf:
+   * `[{ region isAnyOf ["Nowhere"] }, <group>]` — and any query at all over an
+   * empty `rows` — short-circuited to zero matches and answered 200 with no
+   * throw. A result quietly computed from less than the reader asked for is
+   * the one failure these pages exist to argue against, and it does not stop
+   * being that because the result happens to be empty.
+   *
+   * A query is well-formed or it is not, independently of the data; the check
+   * belongs where that question is asked once. `matches()` keeps its branch as
+   * belt-and-braces for any future caller that reaches it directly.
+   */
+  for (const [index, filter] of query.filters.entries()) {
+    if ("children" in filter) rejectFilterGroup(index);
+  }
+
   const filtered = rows.filter((row) =>
     query.filters.every((filter) => matches(row, filter)),
   );

apps/website/app/docs/__tests__/server-data.types.tsx

@@ -7,9 +7,10 @@
  * fence under that heading. The preamble above the first marker is prepended to
  * every region, which is what lets several snippets share one import.
  *
- * `server-data/index.mdx` is deliberately NOT bound here: both of its fences
- * are JSON request and response bodies for `POST /api/docs/rows`, and there is
- * no TypeScript on the page to anchor a region to. Binding it would buy two
+ * `server-data/index.mdx` is deliberately NOT bound here: all three of its
+ * fences are JSON — the request and response bodies for `POST /api/docs/rows`,
+ * and a grouped `filters` payload — and there is no TypeScript on the page to
+ * anchor a region to. Binding it would buy two
  * `UNTRANSCRIBED_FENCES` excuses and nothing else. The route's own shapes are
  * held by the app's typecheck where they are declared.
  */

apps/website/content/docs/server-data/index.mdx

@@ -73,10 +73,62 @@ The response is the three things it takes to describe a result — the rows, how
 }
 ```
 
-Two behaviors are deliberate. Every response waits 500 ms before it is sent, which is long enough that `loading` and `stale` are states you can watch rather than infer. And any filter whose value contains **fail** returns a 500, which is how the [lifecycle page](/docs/server-data/lifecycle) reaches the error phase on demand. A filter the fixture genuinely cannot answer — an unknown column, an operator its column's type cannot use, a missing operand — also returns a 500 with a message saying which, rather than quietly returning every row.
+Two behaviors are deliberate. Every response waits 500 ms before it is sent, which is long enough that `loading` and `stale` are states you can watch rather than infer. And any filter whose value contains **fail** returns a 500, which is how the [lifecycle page](/docs/server-data/lifecycle) reaches the error phase on demand. A filter the fixture genuinely cannot answer — an unknown column, an operator its column's type cannot use, a missing operand, or an AND/OR group — also returns a 500 with a message saying which, rather than quietly returning every row.
 
 That last point is a rule to copy, not a fixture quirk: a backend that ignores a filter it does not understand produces a grid that looks filtered and is not.
 
+## What a filter looks like on the wire
+
+`query.filters` is an array, and it always was. What changed is what an element of it may be: either a **leaf** — the `{ columnId, operator, value }` shape in the request above — or a **group**, `{ "op": "and" | "or", "children": [...] }`, whose children are themselves leaves or groups, to any depth. `filters` is a tree, and it reaches `onQueryChange` and then your endpoint exactly as the grid built it. Nothing flattens, rewrites, or simplifies it on the way out, and [external filter authority](/docs/server-data/query-ownership#what-external-authority-suppresses) does not either — suppression decides what the engine _applies_, never what it _reports_.
+
+```json
+{
+  "query": {
+    "filters": [
+      { "columnId": "total", "operator": "gt", "value": 500 },
+      {
+        "op": "or",
+        "children": [
+          { "columnId": "region", "operator": "isAnyOf", "value": ["North"] },
+          { "columnId": "customer", "operator": "contains", "value": "Labs" }
+        ]
+      }
+    ],
+    "sort": [],
+    "rowGroups": []
+  }
+}
+```
+
+That payload reads `total > 500 AND (region is North OR customer contains "Labs")`, and the four rules that make it mean that are the contract.
+
+**The top-level array is an implicit AND.** It is what a list of filters has always meant — each entry narrows the result further — so groups became _elements_ of that array rather than a new field beside it, and the ordinary one-leaf-per-column case stays the flat list it was. Two consequences follow from the same rule. A payload written before groups existed is still a correct payload. And `"filters": []` constrains nothing, because an AND over no conditions excludes no rows.
+
+**Leaves and groups discriminate on structure, not on a tag.** There is no `kind` field to switch on: a group is the node carrying `op` and `children`, a leaf is the node carrying `columnId` and `operator`. On the client edge, `isPretableFilterGroup` — exported from both `@pretable/core` and `@pretable/react` — makes that test and narrows `PretableFilterNodeFor` to `PretableFilterGroupFor`, so nothing has to hand-roll it. On the server there are no types left to narrow: the query arrived as JSON over HTTP, so write the test yourself, and test for `children`. That is the field a group cannot exist without, and the field a leaf never has.
+
+**An empty group matches every row, under either `op`.** Naive boolean algebra says an empty `or` is false, and that is exactly the wrong answer here: a group with nothing in it is a group someone is part-way through building, and a half-built condition that blanks the grid mid-edit is a bug the reader will read as data loss. So an empty group constrains nothing whichever way it joins — the same answer an empty top-level array gives, for the same reason. Copy that rule into your backend rather than deriving it, or the two sides will disagree about a query the grid considers unfiltered.
+
+**Nesting is bounded at 64 levels.** A tree deeper than that is rejected with the same typed `invalid-query` error an unknown column gets, and the message breadcrumbs the offending node — `query.filters[0].children[3].children[1]` — so you are told where, not just that. The bound exists because every consumer of a captured query recurses over it, and it sits far above any tree a person or a builder UI produces and far below the depth at which any of that recursion is at risk. In practice the grid rejects a too-deep tree before it can publish one, so your endpoint should never see it; bound your own recursion anyway, since a query can also arrive from a saved view, a URL, or a client that is not this grid.
+
+### A server that only understands flat filters has to decide
+
+It cannot be left implicit, because the failure mode of guessing is the one this page keeps warning about: a grid that looks filtered and is not. Three answers are defensible, and which one is right is a property of your backend, not of the grid.
+
+- **Reject.** Answer with an error the moment a group appears, naming it. Cheapest by far, correct at once, and it fails where a reader can see it — an error strip over the rows they had, rather than a result quietly computed from half of what they asked for. This is what the fixture endpoint above does: `applyDocsQuery` in `app/api/docs/rows/dataset.ts` scans `filters` for `children` before it reads a single row, and returns a 500 that says so. Scanning up front rather than inside the row predicate is the part worth copying — a per-row check is reachable only if some row survives the leaves ahead of it, so a leaf matching nothing would have answered an empty result instead of an error. Whether a query is one you can answer is a question about the query, and it is asked once. It is the right posture for a demo, and the right first commit for a real backend too, because it buys you the freedom to implement groups later without having shipped a wrong answer in the meantime.
+- **Flatten — but only when every join is `and`.** A tree whose groups all carry `"op": "and"` is genuinely equivalent to the flat list of its leaves, nesting and all, so collecting them loses nothing. The trap is that this is only true until the first `"op": "or"`, and an `or` is precisely what a user reaches for a group to express. So the flattening has to be _conditional_, and its else-branch has to be reject, never best-effort: a tree containing an `or` cannot be approximated by an AND of its leaves in either direction. Note that an empty group contributes no leaves, which is the correct reading of the rule above.
+- **Implement the recursion.** It is smaller than it sounds — map a leaf to a predicate as you already do, join a group's children with `AND` or `OR`, parenthesize each group, and return the identity `TRUE` for an empty one. The work you actually owe is the parameter binding you owe leaves anyway, over a shape that now nests; a tree of user-supplied operators and operands assembled into SQL by string concatenation is an injection hole whatever its depth.
+
+<Callout type="note">
+  No UI builds a group yet. The header funnel writes, edits, and removes its own
+  column's top-level leaf, exactly as it always has; the tool panel's filter
+  builder is a later sub-project. Until it ships, a group can only reach your
+  endpoint from a query you put there yourself — seeded through the controlled
+  `query` prop, restored from a URL, or loaded from a saved view. The contract
+  is documented now rather than alongside that UI because a shape a server can
+  receive is a contract from the moment it is possible, not from the moment it
+  is common.
+</Callout>
+
 ## Where to go next
 
 - [Query ownership](/docs/server-data/query-ownership) — the `processing` and `query`/`onQueryChange` contract, what external filtering suppresses, and what it deliberately does not.