Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ brain/acquire-wikidata-entities.sh /absolute/private/request-plan.json \
them `ai-moderated` — **AI-changed bytes are never left marked `human`, and bots never
mint `human`** (endorse stays session-only; 403). Humans gate only cross-site pushes.
- **Render-cache keys are manually versioned + load-bearing** (currently `render:v17:`,
`page:home:v10`, `page:articles:v2`, `page:articles-index:v1`, `page:about:v1`,
`page:home:v10`, `page:articles:v3`, `page:articles-index:v1`, `page:about:v2`,
`page:sitemap:v4`, `page:stats:v5`, `page:wikifunctions:v3`,
`page:wikifunctions-verify:v3` — all in `index.ts`) — bump the
prefix whenever output bytes change, or readers get stale HTML for up to 30 days. Asset
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ brain/acquire-wikidata-entities.sh /absolute/private/request-plan.json \
them `ai-moderated` — **AI-changed bytes are never left marked `human`, and bots never
mint `human`** (endorse stays session-only; 403). Humans gate only cross-site pushes.
- **Render-cache keys are manually versioned + load-bearing** (currently `render:v17:`,
`page:home:v10`, `page:articles:v2`, `page:articles-index:v1`, `page:about:v1`,
`page:home:v10`, `page:articles:v3`, `page:articles-index:v1`, `page:about:v2`,
`page:sitemap:v4`, `page:stats:v5`, `page:wikifunctions:v3`,
`page:wikifunctions-verify:v3` — all in `index.ts`) — bump the
prefix whenever output bytes change, or readers get stale HTML for up to 30 days. Asset
Expand Down
52 changes: 37 additions & 15 deletions wiki/src/home.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
// Replaces the static site/out/ pages that build-public.ts used to copy into
// wiki/public/ — articles created via PUT /api/article/:slug now appear
// without a static rebuild. index.ts serves brainLanding() cached in
// RENDER_CACHE ('page:home:v10', TTL 300s), homePage() ('page:articles:v2',
// TTL 300s), aboutPage() ('page:about:v1', TTL 300s), and sitemapXml()
// RENDER_CACHE ('page:home:v10', TTL 300s), homePage() ('page:articles:v3',
// TTL 300s), aboutPage() ('page:about:v2', TTL 300s), and sitemapXml()
// ('page:sitemap:v4', TTL 3600s).
//
// Visual language: warm academic-minimalist — paper background, serif display
Expand Down Expand Up @@ -471,8 +471,8 @@ footer a:hover { text-decoration:underline; }
cross-database pages, and literature — with machine-checkable
provenance on every edge.</li>
<li><a href="/concepts">Wikidata concept links</a> &mdash; every formalized concept keyed
to its Wikidata item, as an open RDF dataset (the basis for a proposed
<em>&ldquo;formalized as (Lean/Mathlib)&rdquo;</em> Wikidata property).</li>
to its Wikidata item, as an open RDF dataset (Wikidata&rsquo;s property for these
links is <em>&ldquo;Mathlib Declaration ID&rdquo;</em>, P14534).</li>
<li><a href="/mcp">Wikibrain MCP</a> &mdash; a remote MCP server any AI agent can
connect to mid-proof: jump between informal mathematics and formal
Mathlib declarations (full REST reference at <a href="/brain/api">/brain/api</a>).</li>
Expand Down Expand Up @@ -565,7 +565,7 @@ export interface AboutStats {
nNotFormalized: number;
}

// The About & method page, served dynamically from D1 (KV 'page:about:v1',
// The About & method page, served dynamically from D1 (KV 'page:about:v2',
// TTL 300s). Replaces the static site/out/about.html asset (retired from
// build_static_pages.py) whose baked stats went stale the day it was built.
// Visual shell reproduced from that page: self-contained inline CSS, warm
Expand Down Expand Up @@ -695,10 +695,10 @@ footer p { margin:4px 0; }
<h2>Wikidata concept links</h2>
<p>Every concept matched to a <em>formalized</em> declaration is also keyed to its
<a href="https://www.wikidata.org/">Wikidata</a> item and published as an open
<a href="/concepts">RDF dataset</a>. This is the basis for a proposed Wikidata
property, <em>&ldquo;formalized as (Lean/Mathlib)&rdquo;</em> — the long-term goal is
for these links to live in Wikidata itself, maintainable by the community and queryable
via SPARQL.</p>
<a href="/concepts">RDF dataset</a>. Wikidata has a property for these links,
<a href="https://www.wikidata.org/wiki/Property:P14534"><em>&ldquo;Mathlib Declaration
ID&rdquo;</em> (P14534)</a>; the long-term goal is for them to live in Wikidata
itself, maintainable by the community and queryable via SPARQL.</p>

<h2>For AI agents: the Wikibrain MCP</h2>
<p>For AI agents, the same graph is queryable as the
Expand All @@ -720,14 +720,36 @@ footer p { margin:4px 0; }
documents where every external link comes from, with its Wikidata property and license.</p>

<h2>Limitations &amp; how to contribute</h2>
<p>Annotations are best-effort and cover a growing sample of WikiProject Mathematics,
<p>Annotations are best-effort and cover a sample of WikiProject Mathematics,
not the whole corpus. A match reflects a judgment that a Mathlib declaration formalizes
a statement; it can be incomplete or wrong, and Mathlib itself evolves, so coverage is
a snapshot rather than a guarantee. Corrections happen right on the site: sign in
(GitHub or Google) to edit annotations in-context on any article, or — without an account — report
a problem from any annotation's tooltip (the ⚑ link). Human edits are patrolled by
moderators, and AI proposals to change a human-authored annotation are queued for
human approval before anything lands.</p>
a snapshot rather than a guarantee. Corrections happen right on the site:
<a href="/login">sign in</a> to edit annotations in-context on any article, or — without
an account — report a problem from any annotation's tooltip (the ⚑ link).</p>

<h2>How edits are moderated</h2>
<p>Annotation edits by people take effect immediately and are shown as human-curated.
They are not screened first: each one is listed, newest first, in
<a href="/recent-changes?kind=edit">Recent changes</a> with an awaiting-patrol marker,
and the <a href="/stats">Stats</a> page shows how many have been patrolled. Every change
to an article&rsquo;s annotations, by a person or by the AI pipeline, is recorded in the
article&rsquo;s history, and patrollers can restore an article&rsquo;s annotations to
any earlier version in that history.</p>
<p>The AI pipeline cannot silently overwrite a human-curated annotation: a pipeline save
that drops or alters one is refused, and the pipeline can file a proposal instead. Under
WikiLean&rsquo;s human-at-boundaries policy (adopted August 2026), those proposals no
longer need a person&rsquo;s approval. Each night a deterministic resolver, which uses no
language model, approves or rejects the proposals that match its fixed rules; the rest
stay pending until a patroller or a later nightly run decides them, or lapse when the
pipeline next saves the article and finds their target annotation removed or already
matching them. When the pipeline approves a change to a human-curated annotation, the
annotation is relabeled: it is then shown as &ldquo;AI-generated &middot; not yet
reviewed&rdquo; and loses the human-curated protection, so later pipeline saves can
change it directly. When a patroller approves a change, the result is shown as
human-curated. People remain the gate wherever WikiLean contributes
to outside projects: the tagging bot opens pull requests to Mathlib on its own, but they
merge only after Mathlib reviewers approve them, and statements are added to Wikidata
only for mappings people have reviewed.</p>

<footer>
<p>Article text from <a href="https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Mathematics">Wikipedia</a>,
Expand Down
4 changes: 2 additions & 2 deletions wiki/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -567,7 +567,7 @@ app.get("/", async (c) => {
});

app.get("/articles", async (c) => {
const cacheKey = "page:articles:v2"; // v2: Recent changes + Stats links in the nav
const cacheKey = "page:articles:v3"; // v3: Wikidata property named (P14534)
const cached = await c.env.RENDER_CACHE.get(cacheKey);
if (cached) return c.html(cached);
const html = homePage(await homeRows(c));
Expand Down Expand Up @@ -613,7 +613,7 @@ app.get("/api/articles", async (c) => {
// static wiki/public/about.html asset, which used to shadow this path with
// stats baked at build time). One aggregate over the D-C5 count columns.
app.get("/about", async (c) => {
const cacheKey = "page:about:v1"; // v1: dynamic port of the static About page
const cacheKey = "page:about:v2"; // v2: moderation copy matches Human-at-boundaries
const cached = await c.env.RENDER_CACHE.get(cacheKey);
if (cached) return c.html(cached);
const db = drizzle(c.env.DB);
Expand Down
5 changes: 4 additions & 1 deletion wiki/src/pages.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1091,7 +1091,10 @@ export function proposalsQueuePage(
const body =
PROPOSALS_CSS +
`<h1>AI proposals</h1>` +
`<p class="muted">The AI may propose updates to human-owned annotations but never applies them ` +
`<p class="muted">The AI pipeline proposes updates to human-owned annotations instead of overwriting them. ` +
`Each night a deterministic resolver decides the proposals that match its fixed rules: it rejects ones that ` +
`change nothing, approves verified Mathlib declaration renames, and approves marking a <code>proof_wanted</code> ` +
`stub as partial. The rest wait here for a patroller ` +
`(<a class="wl-navlink" href="/about">how moderation works</a>). Approving applies the change and keeps the annotation yours; ` +
`rejecting remembers the delta so it is not re-proposed.</p>` +
truncNote +
Expand Down
58 changes: 50 additions & 8 deletions wiki/test/about.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
// replaced the static wiki/public/about.html asset. Pins: 200 with live D1
// counts (COUNT(*) + summed count columns, SUM skipping pre-backfill NULL
// rows), the unified nav + Concepts + MCP links, the rewritten contribution
// copy + licensing notices, and the KV cache (page:about:v1, TTL 300,
// TTL-only invalidation). The harness ASSETS binding always 404s, so a 200
// here also proves the route no longer depends on the retired asset.
// copy, the moderation copy (Human-at-boundaries, ROADMAP 2026-08-06), the
// Wikidata property name (P14534), licensing notices, and the KV cache
// (page:about:v2, TTL 300, TTL-only invalidation). The harness ASSETS binding
// always 404s, so a 200 here also proves the route no longer depends on the
// retired asset.

import { describe, it, expect } from "vitest";
import { setup, get, insertArticle, blockNetwork, type Harness } from "./helpers/harness.js";
Expand Down Expand Up @@ -43,15 +45,54 @@ describe("GET /about", () => {
// The stale "corrections via the source repository" limitation copy is
// replaced by the live contribution loop.
expect(html).not.toContain("Corrections are welcome via the source repository");
expect(html).toContain("sign in");
expect(html).toContain("GitHub or Google");
expect(html).toContain('<a href="/login">sign in</a>');
// Providers are configured per deployment (auth.ts availableProviders);
// the static copy must not promise one that may be absent.
expect(html).not.toContain("GitHub or Google");
expect(html).toContain("⚑");
expect(html).toContain("human approval");
// The Wikidata property exists (P14534); the old proposal-era name is gone.
expect(html).toContain('href="https://www.wikidata.org/wiki/Property:P14534"');
expect(html.replace(/\s+/g, " ")).toContain("<em>&ldquo;Mathlib Declaration ID&rdquo;</em> (P14534)");
expect(html).not.toContain("formalized as (Lean/Mathlib)");
expect(html).not.toContain("basis for a proposed");
// Licensing: article text CC BY-SA, annotations CC0.
expect(html).toContain("https://creativecommons.org/licenses/by-sa/4.0/");
expect(html).toContain("https://creativecommons.org/publicdomain/zero/1.0/");
});

it("describes the Human-at-boundaries moderation policy, not a human gate on proposals", async () => {
const h = setup();
const html = await (await get(h.env, "/about")).text();
const text = html.replace(/\s+/g, " ");
// Copy that no longer matches the code or the ratified policy is gone.
expect(text).not.toContain("queued for human approval");
expect(text).not.toContain("patrolled by moderators");
expect(text).not.toContain("growing sample");
// What actually happens.
expect(html).toContain("<h2>How edits are moderated</h2>");
expect(text).toContain("Annotation edits by people take effect immediately and are shown as human-curated");
expect(text).toContain('in <a href="/recent-changes?kind=edit">Recent changes</a> with an awaiting-patrol marker');
expect(text).toContain('<a href="/stats">Stats</a> page shows how many have been patrolled');
// Revert restores annotations only; the Wikipedia pin is not rolled back.
expect(text).toContain("restore an article&rsquo;s annotations to any earlier version in that history");
expect(text).toContain("a pipeline save that drops or alters one is refused");
expect(text).toContain("those proposals no longer need a person&rsquo;s approval");
expect(text).toContain("deterministic resolver, which uses no language model");
expect(text).toContain("the rest stay pending until a patroller or a later nightly run decides them");
expect(text).toContain("lapse when the pipeline next saves the article");
// index.ts approve path: a bot approval re-labels human → ai-moderated,
// a patroller approval stamps human. The copy states both directions and
// what the relabel means: script.js shows every non-'human' provenance as
// "AI-generated · not yet reviewed", and the anti-clobber guards
// (findLostHuman, _preserve_human) protect only provenance 'human'.
expect(text).toContain("When the pipeline approves a change to a human-curated annotation, the annotation is relabeled: it is then shown as &ldquo;AI-generated &middot; not yet reviewed&rdquo; and loses the human-curated protection, so later pipeline saves can change it directly");
expect(text).not.toContain("stops being shown as human-curated");
expect(text).toContain("When a patroller approves a change, the result is shown as human-curated");
// The bot opens Mathlib PRs unattended; the human gate is Mathlib review.
expect(text).toContain("the tagging bot opens pull requests to Mathlib on its own, but they merge only after Mathlib reviewers approve them");
expect(text).toContain("statements are added to Wikidata only for mappings people have reviewed");
});

it("handles an empty corpus without dividing by zero", async () => {
const h = setup();
h.db.prepare("DELETE FROM revisions").run(); // FK before articles
Expand All @@ -62,10 +103,11 @@ describe("GET /about", () => {
expect(html).not.toContain("NaN");
});

it("is KV-cached under page:about:v1 for 300s (TTL-only invalidation)", async () => {
it("is KV-cached under page:about:v2 for 300s (TTL-only invalidation)", async () => {
const h = setup();
const first = await (await get(h.env, "/about")).text();
expect(h.renderCache.store.has("page:about:v1")).toBe(true);
expect(h.renderCache.store.has("page:about:v2")).toBe(true);
expect(h.renderCache.store.has("page:about:v1")).toBe(false);
insertArticle(h.db, "After_Cache");
const second = await (await get(h.env, "/about")).text();
expect(second).toBe(first);
Expand Down
7 changes: 7 additions & 0 deletions wiki/test/home.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,13 @@ describe("homePage", () => {
expect(html).toContain("https://github.com/Deicyde/WikiLean");
});

it("names the existing Wikidata property (P14534), not the proposal-era one", () => {
const html = homePage([row()]).replace(/\s+/g, " ");
expect(html).toContain("<em>&ldquo;Mathlib Declaration ID&rdquo;</em>, P14534");
expect(html).not.toContain("formalized as (Lean/Mathlib)");
expect(html).not.toContain("basis for a proposed");
});

it("labels the search input for assistive tech", () => {
const html = homePage([row()]);
expect(html).toContain('<label class="sr" for="q">');
Expand Down
4 changes: 4 additions & 0 deletions wiki/test/proposals.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -364,6 +364,10 @@ describe("POST /api/article/:slug (proposals)", () => {
expect(page.status).toBe(200);
const html = await page.text();
expect(html).toContain("AI proposals");
// Human-at-boundaries: the nightly resolver decides some proposals, so the
// intro must not promise the AI never applies them.
expect(html).not.toContain("never applies");
expect(html).toContain("Each night a deterministic resolver decides the proposals that match its fixed rules");
expect(html).toContain(pending(db)[0].proposalId);
expect(html).toContain("wl-prop-approve");
expect(html).toContain("reject: why?");
Expand Down
Loading