Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,18 @@ NEO4J_PASSWORD=your-neo4j-password

PUBLIC_EVAULT_SERVER_URI=http://localhost:4000

# Rate limiting (requests per minute)
# Rate limiting (requests per minute). These are the defaults; limits set from
# the control panel's Rate Limits page override them without a restart.
RATE_LIMIT_PER_PLATFORM=250
RATE_LIMIT_PER_IP=500
# Shared by evault-core and the control panel to manage rate limit rules. The
# admin API is disabled while this is empty. Use a long random value.
RATE_LIMIT_ADMIN_SECRET=""
# Leave empty when clients reach evault-core directly. Behind a reverse proxy,
# set the proxies' addresses (e.g. "10.0.0.5" or "10.0.0.0/8"), or every client
# is rate limited as the proxy's IP. A hop count (e.g. 1) also works, but only
# if evault-core cannot be reached except through the proxy.
EVAULT_TRUST_PROXY=""

REGISTRY_ENTROPY_KEY_JWK='{"kty":"EC","use":"sig","alg":"ES256","kid":"entropy-key-1","crv":"P-256","x":"your-x-value","y":"your-y-value","d":"your-d-value"}'
ENCRYPTION_PASSWORD="your-encryption-password"
Expand Down
62 changes: 62 additions & 0 deletions docs/docs/Infrastructure/eVault.md
Original file line number Diff line number Diff line change
Expand Up @@ -814,6 +814,68 @@ The provisioning layer supports shared tenancy (multiple W3IDs can be provisione
- **Isolation**: All queries filtered by W3ID
- **No Cross-Tenant Access**: Users can only access their own data (unless ACL allows)

## Rate Limiting

The GraphQL and HTTP API (port 4000) is rate limited per minute, first per client IP and then per calling platform. Limits are managed from the control panel's **Rate Limits** page and apply immediately, without a restart.

### How a request is decided

1. A CORS preflight (`OPTIONS`) is always allowed and never counted.
2. The most specific IP rule covering the client decides first: a **blocked** IP gets `403`, an **exempt** IP skips every limit.
3. Otherwise the request counts against the IP's limit: its own rule, or the IP default.
4. If the request carries a valid Registry token, it also counts against that platform's limit: its own rule, or the platform default. A **blocked** platform gets `403`; an **exempt** platform skips only this step.

A request over a limit gets `429` with a `Retry-After` header:

```json
{ "error": "Too Many Requests", "retryAfterSeconds": 42, "scope": "platform" }
```

`scope` is `ip` or `platform`, naming the limit that was hit.

### Rules

| Applies to | Subject | Actions |
| --- | --- | --- |
| IP | A single address or a CIDR range, IPv4 or IPv6 | Set limit, block, exempt |
| Platform | The `platform` value of its Registry token, matched exactly | Set limit, block, exempt |
| Default | Every IP or platform without its own rule | Set limit |

A limit on a range gives each address in it that limit; it is not one shared budget. Block and exempt rules are refused for ranges broader than `/8` (IPv4) or `/32` (IPv6).

### What a platform rule can and cannot do

The Registry signs a token for any platform name it is asked for, so a token proves it is genuine, not who sent it. The limiter verifies the signature and treats an invalid token as no platform at all, but the name itself is self-declared. Two consequences:

- **The IP limit is the ceiling.** A platform limit or exemption never lets a caller exceed its IP limit. To give a platform more headroom, raise or exempt the IPs of its servers as well.
- **A platform block is advisory.** A blocked caller can request a token under another name. Block its IPs to stop it. Likewise, anyone can send a token carrying another platform's name and use up that platform's budget, up to their own IP limit.

### Configuration

| Variable | Purpose |
| --- | --- |
| `RATE_LIMIT_PER_IP` | Default requests per minute per IP (500) until a default is set in the control panel |
| `RATE_LIMIT_PER_PLATFORM` | Default requests per minute per platform (250) until a default is set in the control panel |
| `RATE_LIMIT_ADMIN_SECRET` | Shared with the control panel to manage rules. The admin API is disabled while it is empty |
| `EVAULT_TRUST_PROXY` | Leave empty when clients connect directly. Behind a reverse proxy, the proxies' addresses or CIDR ranges |

Rules are stored in the `evault_rate_limit_rule` table, so run `pnpm --filter evault-core migration:run` when deploying. If the table or the database is unavailable, the limiter keeps the last rules it loaded, or the defaults above if it never loaded any; it never refuses all traffic.

Behind a reverse proxy, set `EVAULT_TRUST_PROXY` to the proxies' addresses or every client is counted as the proxy's IP. A hop count (such as `1`) or `true` is also accepted, but both believe `X-Forwarded-For` from whoever connects: if the eVault port can be reached without going through the proxy, a client can choose the IP it is counted as and sidestep every IP limit and block.

Counters are kept in memory by each eVault process. With several instances behind a load balancer each one counts separately, so the effective limit is the configured value times the number of instances, and the control panel shows the counters of the instance it reached. Rule changes reach every instance within 30 seconds.

### Admin API

The control panel manages rules through the provisioning port (3001), which is not rate limited, so no rule can lock an admin out. Every call needs the `x-shared-secret` header set to `RATE_LIMIT_ADMIN_SECRET`.

| Route | Purpose |
| --- | --- |
| `GET /admin/rate-limits` | Defaults, rules and where they were loaded from |
| `GET /admin/rate-limits/usage` | Live counters per platform and IP for the current window |
| `PUT /admin/rate-limits/rules` | Create or replace the rule for a subject: `{ kind, subject, mode, limitPerMinute?, note? }` with `kind` `ip` or `platform`, `mode` `limit`, `blocked` or `exempt`, and subject `*` for a default |
| `DELETE /admin/rate-limits/rules/:id` | Remove a rule |



## API Examples
Expand Down
7 changes: 7 additions & 0 deletions infrastructure/control-panel/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,13 @@ A SvelteKit-based control panel for monitoring and managing various services and
- **Static admin allowlist**: Access is granted only if the authenticated eName exists in `config/admin-enames.json`
- **No local DB**: Admin authorization is file-based and reloaded from disk when changed

### Rate Limits

- **Per-platform and per-IP limits**: `/rate-limits` sets the eVault API's requests-per-minute limits per platform and per IP or CIDR range, and the defaults for everyone else
- **Block and exempt**: An IP, range or platform can be blocked or exempted
- **Live usage**: Shows the callers closest to their limit in the current window
- **Setup**: Set `RATE_LIMIT_ADMIN_SECRET` to the same value for the control panel and evault-core; `PUBLIC_PROVISIONER_URL` must point at evault-core's provisioning port

## Prerequisites

### Kubernetes Access
Expand Down
163 changes: 163 additions & 0 deletions infrastructure/control-panel/src/lib/server/rate-limits.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
import { env } from '$env/dynamic/private';
import { PUBLIC_PROVISIONER_URL, PUBLIC_REGISTRY_URL } from '$env/static/public';
import { isAdminEName } from '$lib/server/auth/allowlist';
import { error } from '@sveltejs/kit';

export type RateLimitKind = 'platform' | 'ip';
export type RateLimitMode = 'limit' | 'blocked' | 'exempt';
export type LimitSource = 'db' | 'env';

export interface RateLimitRule {
id: string;
kind: RateLimitKind;
subject: string;
mode: RateLimitMode;
limitPerMinute: number | null;
note: string | null;
updatedBy: string | null;
updatedAt: string;
}

export interface RateLimitConfig {
windowSeconds: number;
config: { source: LimitSource; loadedAt: string };
trustProxy: boolean | number | string;
defaults: {
platform: { limit: number; source: LimitSource };
ip: { limit: number; source: LimitSource };
};
rules: RateLimitRule[];
}

export interface UsageRow {
subject: string;
count: number;
rejected: number;
limit: number;
mode: RateLimitMode;
matchedRule: string | null;
resetsInSeconds: number;
}

export interface RateLimitUsage {
generatedAt: string;
windowSeconds: number;
platforms: UsageRow[];
ips: UsageRow[];
truncated: boolean;
}

/** An evault-core answer worth showing the admin as it is. */
export class RateLimitApiError extends Error {
constructor(
message: string,
readonly status: number,
readonly field?: string
) {
super(message);
}
}

/**
* The signed-in admin's eName. The allowlist is re-read here because the
* session cookie outlives a removal from it.
*/
export async function requireAdmin(locals: App.Locals): Promise<string> {
const ename = locals.user?.ename;
if (!ename || !(await isAdminEName(ename))) {
throw error(403, 'Not an admin');
}
return ename;
}

async function call(path: string, init: RequestInit = {}): Promise<Response> {
const secret = env.RATE_LIMIT_ADMIN_SECRET;
if (!secret) {
throw new RateLimitApiError(
'RATE_LIMIT_ADMIN_SECRET is not set for the control panel',
503
);
}
const baseUrl = PUBLIC_PROVISIONER_URL || 'http://localhost:3001';

let response: Response;
try {
response = await fetch(`${baseUrl}${path}`, {
...init,
headers: {
'Content-Type': 'application/json',
'x-shared-secret': secret,
...init.headers
},
signal: AbortSignal.timeout(10000)
});
} catch {
throw new RateLimitApiError('Could not reach evault-core', 502);
}

if (response.ok) return response;
if (response.status === 401) {
throw new RateLimitApiError(
'evault-core rejected RATE_LIMIT_ADMIN_SECRET; it must match on both services',
502
);
}
if (response.status === 503) {
throw new RateLimitApiError('RATE_LIMIT_ADMIN_SECRET is not set for evault-core', 503);
}
const body = await response.json().catch(() => null);
throw new RateLimitApiError(
body?.error ?? `evault-core answered ${response.status}`,
response.status,
body?.field
);
}

export async function getRateLimitConfig(): Promise<RateLimitConfig> {
return (await call('/admin/rate-limits')).json();
}

export async function getRateLimitUsage(): Promise<RateLimitUsage> {
return (await call('/admin/rate-limits/usage')).json();
}

export async function saveRateLimitRule(
rule: Record<string, unknown>,
updatedBy: string
): Promise<RateLimitRule> {
const response = await call('/admin/rate-limits/rules', {
method: 'PUT',
body: JSON.stringify({ ...rule, updatedBy })
});
return (await response.json()).rule;
}

export async function deleteRateLimitRule(id: string, removedBy: string): Promise<void> {
await call(`/admin/rate-limits/rules/${encodeURIComponent(id)}`, {
method: 'DELETE',
headers: { 'x-admin-ename': removedBy }
});
}

/** Platform names the Registry lists, to offer before they have traffic. */
export async function getKnownPlatforms(): Promise<string[]> {
if (!PUBLIC_REGISTRY_URL) return [];
try {
const response = await fetch(new URL('/platforms', PUBLIC_REGISTRY_URL), {
signal: AbortSignal.timeout(5000)
});
if (!response.ok) return [];
const platforms: unknown[] = await response.json();
return platforms.filter((p): p is string => typeof p === 'string' && p.trim() !== '');
} catch {
return [];
}
}

/** The JSON answer for a failed call, keeping evault-core's own message. */
export function errorResponse(err: unknown): Response {
if (err instanceof RateLimitApiError) {
return Response.json({ error: err.message, field: err.field }, { status: err.status });
}
throw err;
}
1 change: 1 addition & 0 deletions infrastructure/control-panel/src/routes/+layout.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
{ label: 'Monitoring', href: '/monitoring' },
{ label: 'Actions', href: '/actions' },
{ label: 'Notifications', href: '/notifications' },
{ label: 'Rate Limits', href: '/rate-limits' },
{ label: 'Visualizer', href: '/visualizer' }
];

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import {
errorResponse,
getKnownPlatforms,
getRateLimitConfig,
requireAdmin
} from '$lib/server/rate-limits';
import { json } from '@sveltejs/kit';
import type { RequestHandler } from '@sveltejs/kit';

export const GET: RequestHandler = async ({ locals }) => {
await requireAdmin(locals);
try {
const [config, knownPlatforms] = await Promise.all([
getRateLimitConfig(),
getKnownPlatforms()
]);
return json({ ...config, knownPlatforms });
} catch (err) {
return errorResponse(err);
}
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import { errorResponse, requireAdmin, saveRateLimitRule } from '$lib/server/rate-limits';
import { json } from '@sveltejs/kit';
import type { RequestHandler } from '@sveltejs/kit';

export const PUT: RequestHandler = async ({ locals, request }) => {
const admin = await requireAdmin(locals);
const body = await request.json().catch(() => null);
if (!body || typeof body !== 'object' || Array.isArray(body)) {
return json({ error: 'Expected a JSON object' }, { status: 400 });
}
try {
return json({ rule: await saveRateLimitRule(body, admin) });
} catch (err) {
return errorResponse(err);
}
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { deleteRateLimitRule, errorResponse, requireAdmin } from '$lib/server/rate-limits';
import type { RequestHandler } from '@sveltejs/kit';

export const DELETE: RequestHandler = async ({ locals, params }) => {
const admin = await requireAdmin(locals);
try {
await deleteRateLimitRule(params.id ?? '', admin);
return new Response(null, { status: 204 });
} catch (err) {
return errorResponse(err);
}
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { errorResponse, getRateLimitUsage, requireAdmin } from '$lib/server/rate-limits';
import { json } from '@sveltejs/kit';
import type { RequestHandler } from '@sveltejs/kit';

export const GET: RequestHandler = async ({ locals }) => {
await requireAdmin(locals);
try {
return json(await getRateLimitUsage());
} catch (err) {
return errorResponse(err);
}
};
Loading
Loading