Repository navigation
A write token may set up a base - #1244
Conversation
Five routes take a personal token with write scope as well as a session, acting as the token's owner: creating a base in a workspace, creating a source, reading a source's push key, importing an ontology, and deciding a kind word. Listing a workspace's bases takes any token. The person's role is still the ceiling, a read token is refused with token_read_only, and a token limited by kb_ids reaches only those bases and creates none (token_kb_out_of_scope). One extractor, SessionOrToken, takes either credential; AuthUser still reads every bearer as a JWT. 0014 gains decision 6. Closes deeplethe#1243 Signed-off-by: Bogdan Szabo <bogdan@re-cinq.com>
## Why #2767 mirrors each repo's spec and ADR chain into Utopia, but a person has to run the setup route once per repo with their Utopia session. The ask was that Lore creates the knowledge base itself when it is missing. Utopia's setup routes took a session only; deeplethe/utopia#1244 lets a write-scoped personal token call them, acting as its owner. It is open upstream and ships in our build `ghcr.io/re-cinq/utopia:0.1.0-re.2` (re-dev). This is the Lore side, off by default. ## What Changed - `withSession(jwt)` becomes `withCredential(bearer)`: the same requests with a session JWT or a `utp_pat_…` write token. - With `UTOPIA_API_URL`, `UTOPIA_TOKEN` and `UTOPIA_WORKSPACE_ID` set, the first spec or ADR ingest of a repo with no stored source runs the setup under the token (`apps/lore-api/src/work/utopia/utopia-auto-setup.ts`) and stores the key and settings through `storeUtopiaSetup`, the one writer the setup route now shares. Two deltas for one repo in a process share one setup; across processes the base is found by name and the per-repo upsert keeps the last saved key. A failed setup is a mirror failure: logged, counted, never raised to the ingest. - The setup route is unchanged for deployments without a token. - Infra behind `enable_utopia_auto_setup` (default false) and `utopia_lore_workspace_id`: GSM container `lore-utopia-lore-api-token`, ExternalSecret `utopia-token` in `lore-api`, `seed-secrets.sh` OPTIONAL entry, chart env `UTOPIA_WORKSPACE_ID` and `UTOPIA_TOKEN` (secretKeyRef) only when the flag is on, so the chart never references a secret that does not exist. A validation refuses the flag without `enable_utopia` and a workspace id. - Spec FR5.5 and FR6.1–6.4, an ADR-052 amendment, `docs/managing-secrets.md` (minting and rotating the token), CLAUDE.md. ## Testing Client and `sourceToMirror` tests against a local HTTP Utopia: first ingest sets the repo up and stores the key; two concurrent calls (`re-cinq/lore`, `re-cinq/Lore`) make one setup; a stored source makes no calls; no token means nothing happens; a 403 stores nothing and the next call succeeds. libs/shared 3226 and lore-api 1596 tests pass; `tsc` clean everywhere; `terraform fmt`/`validate` pass; the chart renders with the flag off, on, and on with the token. To turn on, once Utopia runs 0.1.0-re.2: mint the token in Utopia, seed `lore-utopia-lore-api-token`, set both variables, `terraform apply` from main. Refs #2766
## Why The image Utopia runs was pinned only in the git-ignored `terraform.tfvars`, so main never recorded which build is deployed, and the apply on 2026-10-15 kept running `0.1.0-re.1` because a local file still named it. `0.1.0-re.2` carries what the Lore mirror needs: deeplethe/utopia#1244 (a write token sets up a base, for #2768's auto-setup) and #1245 (a pushed phrase equal to a declared relation is typed without the model). ## What Changed - `variables.tf` defaults: `utopia_image_repository = "ghcr.io/re-cinq/utopia"`, `utopia_image_tag = "0.1.0-re.2"`, `utopia_image_digest = "sha256:936141d3…"` (the pushed image, built from gedaiu/utopia `re-dev` at tag `v0.1.0-re.2`). The descriptions say what the build carries and that a bump is a PR here. - `terraform.tfvars.example` no longer sets the tag; it says the pin lives in the defaults and tfvars overrides only to test a build. ## Testing `terraform fmt -check` and `terraform validate` pass. The image boots on a fresh database, runs migration 0113 and answers health; the full Utopia suite passed on the tagged commit (1,545 tests). Refs #2766
WaylandYang
left a comment
There was a problem hiding this comment.
Thank you, this is careful work: the token only narrows, the role checks are unchanged, and the read/expired/revoked/kb_ids cases are all covered by tests that run in CI.
One change before merge: please limit what a token can create in POST /kbs/{id}/sources to the push kinds (statements, and api if you need it), and refuse other kinds from a token with a coded 403. As it stands a write token can create url/rss/custom sources, which the server starts fetching immediately (sources_routes.rs:111), with stored credentials for custom. That turns a leaked CI token from "can read and propose memories" into "can make the server call any address on a schedule", and #1243 only needs the statements source. A session keeps creating every kind. One test case for the refusal is enough.
Also, please put the token id in the audit metadata on the rows these routes already write (source.created, alignment.kind_word_decided, ontology import). 0014 decision 5 relies on knowing which token acted; without it, an admin can't tell what a leaked token did before revoking it. A line in decision 6 noting that a system admin's write token acts as Owner in every base would also help the next reader.
A write token may create only a statements source; any other kind is refused with token_source_kind, since url, rss and custom sources make the server fetch on a schedule. source.created, alignment.kind_word_decided and ontology.imported carry detail.token_id when a token acted, so an admin can see what a token did before revoking it. Decision 6 of 0014 says so, and that a system admin's write token acts as Owner in every base. Signed-off-by: Bogdan Szabo <bogdan@re-cinq.com>
|
Thanks, both done in 0ed8e9b.
Local runs: fmt and clippy ( |
Pins Utopia `0.1.0-re.4` (`sha256:0697b93d1c976b02409e63f08ed9b1706d4806b31466208d8da61e2464b9e02c`) in the committed defaults of `infra/terraform/variables.tf`. `re.4` is `re.3` (re-dev `bc23e69`, which includes #1250) plus the fixes the maintainer asked for in their review of: - deeplethe/utopia#1244: a write token creates only `statements` sources (anything else gets a 403 `token_source_kind`), and the audit rows for source creation, kind-word decisions and ontology imports record the token id. - deeplethe/utopia#1245: a declared state relation is left to the aligner instead of being bound by the ontology. Lore pushes only `statements` sources and declares no state relations, so neither fix changes what the mirror does. There is no new migration: 113 is still the last one. Before tagging, the merged re-dev (`26ea10b`) passed fmt, clippy and the `utopia-server` / `utopia-store` tests. The image was boot-tested on an empty database: health returned 200 and migrations ran to 113. Takes effect with `terraform apply` from main.
WaylandYang
left a comment
There was a problem hiding this comment.
Thank you, both points are addressed exactly as asked: a token now creates only statements sources and is refused with token_source_kind before anything is written, sessions are unaffected, and the three audit rows carry detail.token_id so a leaked token's actions can be traced before it is revoked. Decision 6 reads well with the system-admin sentence. Merging.
Closes #1243.
What happened
A backend that mirrors many repositories into Utopia needs a base and a
statementssource per repository. The push is the source key's job (0054), but creating the base and the source took a session, so the backend either held someone's seven-day JWT or a person did the setup by hand for every repository.What changed
Five routes now also take a personal token with
writescope, acting as the person who issued it:POST /workspaces/{id}/kbs,POST /kbs/{id}/sources,GET /kbs/{id}/sources/{sid}/token,POST /kbs/{id}/ontology/importsandPOST /kbs/{id}/review/alignment/kind-words/{word}.GET /workspaces/{id}/kbstakes any token, so a client can find a base by name before it creates one.SessionOrToken, resolves either credential to the sameUser. Only these routes and document content (0052, whereDocumentReaderwas a near-copy of it and is folded in) use it;AuthUserstill reads every bearer as a JWT, so every other route stays session-only.require_writefirst and then the role check it already had: a read token is refused with 403token_read_only, a token limited bykb_idsreaches only those bases and creates none (403token_kb_out_of_scope), and the owner's role is still the ceiling.AppError::CodedForbiddencarries those codes, the same shape asCodedConflict, so a client can tell "use another token" from "your role is not enough".0014 gains decision 6 with the reasons it keeps (a living person is the actor, scope from the token, writing must be ticked, no base-level machine tokens or service accounts) and what is not done. Its status line also said
can_writewas hard-coded false, which #318 had fixed; a dated note corrects it.Tests
api::token_setup_tests, against a real database: a workspace admin's write token creates a base and a source, reads the push key, passes the import and kind-word checks, and the audit rows name the admin, while an expired and a revoked token from the same person get 401; a read token lists bases and is refused withtoken_read_onlyon all five writing routes; a viewer's write token creates no base, and a token limited to base A creates a source in A but not in B and no base, and lists only A.cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warningsandcargo test --workspacepass withUTOPIA_TEST_REQUIRE_DB=1(utopia-server 634 passed, utopia-store 306). The web UI needs no change.