Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ State / architecture / decisions / "why": Obsidian Vault, `AINode` (cluster ops:
- **Every node-to-node request AINode makes carries the FLEET KEY, and there is one helper that puts it there** (`ainode/auth/fleet.py`: `fleet_headers(app)` off a running app, `fleet_key_headers(secret)` off a config). The key is `HMAC-SHA256(cluster_secret, "ainode-fleet-key-v1")`, so every node holding the secret computes the same one, the join flow already distributes it, rotation follows the secret, and NOTHING new is written to disk. The middleware accepts it as the caller id `fleet` (`auth/middleware.py::identify_caller`), reading the secret LIVE per request, so a node whose secret differs refuses its would-be peers exactly as it drops their datagrams. Auth was enabled-able and unusable on a cluster before this: a node with `auth.enabled` answered 401 to its own fan-outs, so the fleet ran open. Add a peer call and you add the header: an ENGINE port is not a peer call in this sense (the inference proxy, the capability probes and the embeddings route talk to a vLLM container, which never sees this middleware), and `POST /api/cluster/join` is keyless by construction. `tests/test_fleet_auth.py` WALKS THE SOURCE for peer URLs and fails on one whose function does not name the helper, with an exempt list that has to state a reason.
- **A fresh install requires a key, and an update never changes an installed node's access control** (`scripts/install.sh`). The installer mints one key on a node with no `config.json`, stores the SHA-256 the way `AuthConfig` does, prints the plaintext ONCE in a box, and the summary reads "API protected, one key"; `AINODE_AUTH=off` keeps the old open behaviour and prints what that choice means. Both gates matter: minting on an existing home would lock out every client the operator already pointed at the node, with a key they never saw. Because of this, `ainode update` verifies the running release on `/api/health` and not `/api/status` (health is the one keyless route, which is why it carries `version`): reading a keyed route there made every update on a protected node pull, pin, restart and then report that it had not applied.
- **`auth.json` and `config.json` are 0600, written temp-then-replace, and re-read when they change.** Both carry credentials (`config.json` holds `cluster_secret` and may hold `hf_token`) and both were written under the default umask as root. `AuthConfig.reload_if_changed()` stats the store per request the way `ClusterSecret` stats the config per datagram, which is what makes `ainode auth enable|disable|key create|key revoke` LIVE on a running node (it used to need a restart nobody was told to do, so the CLI said "Auth enabled" about a node that went on answering everybody). A file that cannot be stat'ed or parsed keeps the state in memory: the tolerant direction is never "let everybody in". A credential-shaped config key goes in `api/server.py::SCRUBBED_CONFIG_KEYS` and is masked by `ainode config`, or it leaves the node in a `GET /api/config`.
- **A person logs in, a machine presents a key, and the two stores sit side by side** (`ainode/auth/accounts.py`, `auth/session_routes.py`, #261). `users.json` is 0600, written temp-then-replace and re-read per request exactly as `auth.json` is, because `ainode auth user add`, a password change and a revoked session all have to land on a running node. The ONLY crypto is the standard library's: `hashlib.scrypt` (n=2**14, r=8, p=1) over a per-password salt, and a session token stored as its SHA-256 alone, so the file a backup or a support bundle picks up logs nobody in and a login page is not the reason the image grows an argon2 wheel. **A session does not expire** (in until you log out), which makes REVOCATION the whole control and it has to be complete: a password change, a disable and a removal each take that account's open sessions with them, and `session_for_token` refuses one whose account has since been removed or disabled. **A cookie-authenticated request whose method is not GET, HEAD or OPTIONS must carry `X-AINode-Client: dashboard`**, and that header is the whole CSRF defence: a browser attaches a cookie to a cross-site request on its own, and a custom header cannot be set cross-origin without a preflight this node never approves, so dropping the rule would make every node an operator is logged into writable by any page on the internet. A Bearer token is tried FIRST and the cookie only when no key matched, so automation is untouched and a stale key in a browser does not cost a good session its request. Administration (`/api/auth/users*`, `/api/auth/keys*`, `/api/auth/enable|disable`) goes through `accounts.require_admin`: an admin session, an operator key, or the fleet key, never a member; the two replication routes are the FLEET key alone because they move password hashes; and the last ENABLED admin cannot be removed or disabled, enforced in the store (`UsersStore.is_last_admin`) so the CLI cannot get around it. `/api/auth/login` and `/api/auth/me` are the newest entries in `SKIP_PATHS`, and adding one there means naming it in the middleware docstring, `api/cluster_join.py`'s docstring, the README's keyless paragraph and the dashboard's API access panel, because four tests assert each of those.
- **A discovery announcement is SIGNED when `cluster_secret` is set, and a node that has one drops what it cannot verify** (`discovery/signing.py`). The signature is one extra top-level key beside the payload's own fields, never a wrapper around them, so a peer that has never heard of it drops the unknown key in `from_json` and stays in the cluster view. Both directions read the secret per datagram through `ClusterSecret`, so rotating it is a `config.json` edit and not a fleet-wide restart: never capture the value at startup. A node with NO secret behaves exactly as it did and says so once per process, because nothing on the fleet sets one and a mandatory-signing release would partition every existing cluster on upgrade.
- **The discovery port has one home: `core/config.py::DEFAULT_DISCOVERY_PORT` (5679).** `NodeConfig`, both discovery classes and the installer read it from there. They each carried their own literal, and the dataclass said 5678 while the installer, the fleet and the docs said 5679, so a source install listened where nobody spoke and vanished from every cluster view with nothing logged (#181). The port, the cluster id and whether the wire is signed are logged together at startup.
- **`ainode_version` travels on the announcement and into every view that lists nodes** (`/api/nodes`, `/api/cluster/resources`, `/api/version/check`, `/api/cluster/update-status`). A peer that announces none reports `""` and is counted as unknown: filling it in with the local version is how a split fleet goes on looking like a healthy one (#171).
Expand Down
9 changes: 6 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1006,8 +1006,8 @@ belongs to the node whose files it writes.
bytes, stored on the master as a SHA-256 hash with an expiry (30 minutes, `--ttl`) and
one use, and it is the joiner's only credential: `POST /api/cluster/join` answers
without an API key, because a node that has not joined cannot hold this cluster's key
yet. It is one of five keyless paths, with `/api/health`, `/api/auth/status`,
`/api/cluster/endpoint` and the static shell. A wrong, an expired and a spent token all get the same 403,
yet. It is one of seven keyless paths, with `/api/health`, `/api/auth/status`,
`/api/auth/login`, `/api/auth/me`, `/api/cluster/endpoint` and the static shell. A wrong, an expired and a spent token all get the same 403,
and the handler allows five attempts a minute per source address. The joining side
writes `cluster_id`, `cluster_secret`, `cluster_role`, `distributed_mode`,
`master_address` and `discovery_port` into `config.json` and touches nothing else,
Expand Down Expand Up @@ -1133,7 +1133,10 @@ With auth on, every path under `/api` and `/v1` wants the key, with these delibe
exceptions: the static shell (`/` and `/static/*`), because it is what asks for the
key; `/api/health`, because a liveness probe has none, which is also why `ainode
update` verifies a release there; `/api/auth/status`, so the UI can say a key is
wanted instead of rendering blank; `/api/cluster/endpoint`, which carries names,
wanted instead of rendering blank; `/api/auth/login`, because the caller with no
credential is exactly who knocks on it, and `/api/auth/me`, which answers
`{"user": null}` to a caller it does not recognise so the dashboard can draw the
login page instead of guessing; `/api/cluster/endpoint`, which carries names,
addresses and ports so a client stranded by its own node can find another; and `POST
/api/cluster/join`, because a node joining this cluster cannot hold this cluster's key
yet. That last one takes a single-use expiring join token instead, and the handler
Expand Down
5 changes: 3 additions & 2 deletions ainode/api/cluster_join.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
"""The two join routes: the one a joiner calls, and the one this node's UI calls.

``POST /api/cluster/join`` runs on the MASTER. It is the only route besides
``/`` and ``/static/*``, ``/api/health``, ``/api/auth/status`` and
``/api/cluster/endpoint`` that answers without an API key, and the
``/`` and ``/static/*``, ``/api/health``, ``/api/auth/status``,
``/api/auth/login``, ``/api/auth/me`` and ``/api/cluster/endpoint`` that answers
without an API key, and the
reason is structural: the node calling it has not joined yet, so it cannot hold
this cluster's key. The join token IS the credential (32 random bytes, stored
hashed, single use, expiring), and everything that follows from that lives here:
Expand Down
11 changes: 11 additions & 0 deletions ainode/api/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@
auth_middleware,
is_authenticated,
)
from ainode.auth.accounts import UsersStore
from ainode.auth.api_routes import register_auth_routes
from ainode.auth.session_routes import register_session_routes
from ainode.ratelimit.middleware import (
RateLimitConfig,
RateLimiter,
Expand Down Expand Up @@ -125,6 +127,11 @@ def create_app(
config = NodeConfig()

auth_config = AuthConfig.load()
# The accounts beside the keys (#261). Built here so the middleware, the
# login routes and the CLI all read one store, and loaded the same way
# auth.json is: a malformed file raises at boot rather than letting the node
# come up with no accounts and an operator who thinks there are some.
users_store = UsersStore.load()

# Order matters. The rate limiter is LAST, so it runs innermost: by then the
# auth middleware has stamped the API key id, which is what the limiter keys
Expand All @@ -150,6 +157,7 @@ def create_app(

app["config"] = config
app["auth_config"] = auth_config
app["users_store"] = users_store
# Per-client limits on /v1. Off unless config.json says otherwise, so a node
# that never heard of the block behaves exactly as it did before.
app["rate_limiter"] = RateLimiter(config=RateLimitConfig.from_config(config))
Expand Down Expand Up @@ -294,6 +302,9 @@ def create_app(
register_model_routes(app, models_dir=config.models_dir)

register_auth_routes(app)
# The login half of auth, beside the key half: one door for people, one for
# machines (ainode/auth/session_routes.py).
register_session_routes(app)

# --- Metrics routes ------------------------------------------------------
register_metrics_routes(app, collector)
Expand Down
12 changes: 11 additions & 1 deletion ainode/auth/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1,11 @@
"""AINode authentication — optional API key auth for network-exposed instances."""
"""AINode authentication: API keys for machines, logins for people.

Two credentials live here, plus the one the fleet derives for itself:

* ``middleware.AuthConfig`` and ``api_routes`` are the API keys (``auth.json``),
which is what the bench, the desktop app and ``curl`` present.
* ``accounts.UsersStore`` and ``session_routes`` are the named accounts and login
sessions (``users.json``), which is what a person presents (#261).
* ``fleet`` derives a node-to-node key from ``cluster_secret``, so a cluster can
run with auth on everywhere without a second credential to distribute.
"""
Loading
Loading