-
Notifications
You must be signed in to change notification settings - Fork 1.2k
feat(plugins): load local plugins and expose an upstream rewrite slot #5896
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
089f9f0
feat(plugins): load local plugins and expose an upstream rewrite slot
halysondev ca210f9
fix(plugins): rewrite at the physical send and harden plugin setup
halysondev 0dafeba
fix(plugins): direct loopback HTTP, per-exchange WS rewrite, stricter…
halysondev 8515237
fix(plugins): trust the whole plugin path and settle rewritten WS routes
halysondev 6bfe9af
fix(plugins): keep frame-carried Codex turn headers out of WS rewrites
halysondev 4e6409f
docs(plugins): exclude per-turn headers from the WebSocket reuse wording
halysondev File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| --- | ||
| title: Local Plugins | ||
| description: Load your own code into the proxy at startup to rewrite provider sends, for example to put a local compression proxy in front of providers. | ||
| --- | ||
|
|
||
| A local plugin is a TypeScript or JavaScript file that `ocx start` loads before the proxy begins | ||
| serving. It can see every provider send just before it leaves the process and redirect it or add | ||
| headers — enough to place a local sidecar (a compression proxy, a recorder) in front of providers | ||
| without changing opencodex itself. | ||
|
|
||
| Plugins are local to one install. opencodex does not download, update or sign them. | ||
|
|
||
| ## Where plugins live | ||
|
|
||
| Put plugin files in `plugins/` inside the opencodex home (`~/.opencodex/plugins/`, or | ||
| `$OPENCODEX_HOME/plugins/` when that variable is set): | ||
|
|
||
| ```text | ||
| ~/.opencodex/plugins/ | ||
| my-sidecar.ts | ||
| ``` | ||
|
|
||
| - Files ending in `.ts`, `.js` or `.mjs` are loaded in name order. | ||
| - Names starting with `.` or `_`, and `*.d.ts`, are ignored — rename a plugin to `_my-sidecar.ts` | ||
| to switch it off. | ||
| - The directory is optional. Without it nothing is loaded. | ||
| - A plugin runs inside the proxy with your credentials, so opencodex refuses a plugin file or a | ||
| `plugins/` directory that is owned by another user or writable by group or others, and refuses | ||
| symbolic links. Every directory above `plugins/`, up to `/`, must also be owned by you or root and | ||
| not writable by group or others, unless it is sticky like `/tmp`. Fix permissions with | ||
| `chmod go-w ~/.opencodex/plugins ~/.opencodex/plugins/*`; on systems whose default umask is | ||
| `002`, check the parent directories too. | ||
| - On Windows these owner and permission checks are not performed; only regular files are loaded. | ||
| Keep the `plugins/` directory writable by your account only. | ||
|
|
||
| Restart the proxy after adding, changing or removing a plugin (`ocx service restart`, or stop and | ||
| start `ocx start`). Each loaded plugin prints a `Plugin loaded: <name>` line at startup; a skipped | ||
| plugin prints the reason. | ||
|
|
||
| To start once without plugins, set `OCX_PLUGINS=0`. | ||
|
|
||
| ## Writing a plugin | ||
|
|
||
| A plugin default-exports an object with an optional `name` and a `setup` function. `setup` receives a | ||
| context. An asynchronous `setup` has five seconds to finish; plugins run in the proxy's own thread, so | ||
| a `setup` that blocks synchronously cannot be interrupted and delays startup until it returns. A | ||
| `setup` that times out is not stopped either: servers or timers it already started keep running, so | ||
| start long-lived resources only after the work that can fail. | ||
|
|
||
| ```ts | ||
| interface UpstreamTarget { | ||
| url: string; // absolute upstream URL; assign a new one to redirect | ||
| headers: Headers; // outbound headers, including credentials — never log them | ||
| readonly transport: "http" | "websocket"; | ||
| } | ||
|
|
||
| export default { | ||
| name: "my-sidecar", | ||
| setup(ctx: { | ||
| log(message: string): void; | ||
| registerUpstreamRewriter(rewrite: (target: UpstreamTarget) => void): void; | ||
| onShutdown(teardown: () => void): void; | ||
| }) { | ||
| ctx.registerUpstreamRewriter(target => { | ||
| const upstream = new URL(target.url); | ||
| if (!upstream.pathname.endsWith("/chat/completions")) return; | ||
| target.url = `http://127.0.0.1:9000${upstream.pathname}${upstream.search}`; | ||
| target.headers.set("x-original-origin", upstream.origin); | ||
| }); | ||
| }, | ||
| }; | ||
| ``` | ||
|
|
||
| The context also carries `name`, `configDir` (the opencodex home) and `pluginDir`. | ||
|
|
||
| Plugins cannot import opencodex modules — in the packaged binary they are not on disk. Declare the | ||
| small interfaces you need locally, as above. | ||
|
|
||
| ## How rewrites behave | ||
|
|
||
| - The rewriter runs synchronously on every provider send over HTTP and on the Codex WebSocket | ||
| connection, after opencodex has picked the transport. Keep it fast; do network checks (health | ||
| probes) in the background and read a cached result in the rewriter. | ||
| - A send redirected to a loopback address (`127.0.0.1`, `::1`, `localhost`) connects directly, over | ||
| HTTP and over the Codex WebSocket, ignoring provider proxies and `HTTP_PROXY`: a proxy elsewhere | ||
| cannot reach this machine's loopback. Any other destination follows the normal egress settings | ||
| (including `NO_PROXY`), evaluated against the rewritten URL, on both transports. | ||
| - The Codex WebSocket rewriter runs for every turn, before an idle pooled socket is reused, and a | ||
| socket is only reused for the same destination and the same rewritten headers, apart from the two | ||
| per-turn headers `x-codex-turn-state` and `x-codex-turn-metadata`. Those travel inside each | ||
| request frame and may differ between exchanges on one socket; changes a rewriter makes to them | ||
| are discarded. A plugin that starts or stops redirecting takes | ||
| effect on the next turn. | ||
| - It runs after opencodex has chosen the provider, account and route, so it does not change routing, | ||
| account selection, retries or request logs. | ||
| - A redirected send goes to the host you chose. That host sees the request exactly as the provider | ||
| would, credentials included. | ||
| - If a rewriter throws, opencodex undoes that rewriter's edits to the send and disables it for the | ||
| rest of the process. Edits made by rewriters that ran before it are kept, so the send goes out as | ||
| those left it (unmodified when it is the only plugin). If `setup` throws or times out, the plugin is skipped, | ||
| anything it registered is removed, and later registration attempts from it are ignored; other | ||
| plugins and the proxy start normally. | ||
| - A plugin directory that exists but cannot be read (for example, wrong permissions) is reported at | ||
| startup rather than treated as empty. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,247 @@ | ||
| /** | ||
| * Local plugin loader. | ||
| * | ||
| * `ocx start` imports every `*.ts`, `*.js` and `*.mjs` file in `$OPENCODEX_HOME/plugins/` | ||
| * before the server binds, so a plugin's hooks are in place for the first request. A missing | ||
| * directory means no plugins and no work. `OCX_PLUGINS=0` disables loading for one start. | ||
| * | ||
| * A plugin is a module whose default export is `{ name?, setup(ctx) }`. It runs in the proxy | ||
| * process with the operator's credentials, so the loader accepts only files owned by the | ||
| * current user that no other user can write — the same trust boundary as `config.json`. | ||
| * Plugins cannot import ocx internals (a compiled binary keeps them inside `$bunfs`); they | ||
| * receive everything they may use through `OcxPluginContext`. | ||
| * | ||
| * Failures are contained: a plugin that throws, times out or has the wrong shape is reported | ||
| * and skipped, its context stops accepting registrations, and the remaining plugins and the | ||
| * proxy start normally. The setup deadline bounds setup that yields to the event loop; plugins | ||
| * run in the proxy's own thread, so synchronous work that never yields cannot be interrupted. | ||
| */ | ||
|
|
||
| import { lstatSync, readdirSync, realpathSync } from "node:fs"; | ||
| import { basename, dirname, join } from "node:path"; | ||
| import { pathToFileURL } from "node:url"; | ||
| import { getConfigDir } from "../config/paths"; | ||
| import { registerOptionalShutdownHook } from "../lib/optional-shutdown-hooks"; | ||
| import { registerUpstreamRewriter, type UpstreamRewriter } from "./upstream-hooks"; | ||
|
|
||
| export type { UpstreamRewriter, UpstreamTarget, UpstreamTransport } from "./upstream-hooks"; | ||
|
|
||
| export interface OcxPluginContext { | ||
| /** The plugin's own name, as reported in logs. */ | ||
| readonly name: string; | ||
| /** `$OPENCODEX_HOME`, for plugins that keep state next to the proxy's. */ | ||
| readonly configDir: string; | ||
| /** `$OPENCODEX_HOME/plugins`. */ | ||
| readonly pluginDir: string; | ||
| log(message: string): void; | ||
| /** See `src/plugins/upstream-hooks.ts`. Called synchronously on every provider send. */ | ||
| registerUpstreamRewriter(rewrite: UpstreamRewriter): void; | ||
| /** Runs once when the proxy shuts down. Must not throw or block. */ | ||
| onShutdown(teardown: () => void): void; | ||
| } | ||
|
|
||
| export interface OcxPlugin { | ||
| name?: string; | ||
| setup(context: OcxPluginContext): void | Promise<void>; | ||
| } | ||
|
|
||
| export interface PluginLoadResult { | ||
| file: string; | ||
| name: string; | ||
| loaded: boolean; | ||
| error?: string; | ||
| } | ||
|
|
||
| const PLUGIN_EXTENSIONS = [".ts", ".js", ".mjs"]; | ||
| const SETUP_TIMEOUT_MS = 5_000; | ||
|
|
||
| export function pluginDirectory(): string { | ||
| return join(getConfigDir(), "plugins"); | ||
| } | ||
|
|
||
| /** A missing directory is "no plugins"; any other read failure propagates to be reported. */ | ||
| function listPluginFiles(dir: string): string[] { | ||
| let entries: string[]; | ||
| try { | ||
| entries = readdirSync(dir); | ||
| } catch (error) { | ||
| if ((error as NodeJS.ErrnoException).code === "ENOENT") return []; | ||
| throw error; | ||
| } | ||
| return entries | ||
| .filter(entry => !entry.startsWith(".") && !entry.startsWith("_") && !entry.endsWith(".d.ts")) | ||
| .filter(entry => PLUGIN_EXTENSIONS.some(extension => entry.endsWith(extension))) | ||
| .sort() | ||
| .map(entry => join(dir, entry)); | ||
| } | ||
|
|
||
| /** | ||
| * Null when `path` is safe to trust, otherwise the reason it is refused. `lstat` is used so a | ||
| * symbolic link is judged as a link — and refused — rather than as the file it points to: a | ||
| * link to a file you own would otherwise pass the owner and mode checks. Windows has no | ||
| * POSIX owner or mode bits, so there only the file type is checked. | ||
| */ | ||
| function trustError(path: string, kind: "file" | "directory"): string | null { | ||
| let stats: ReturnType<typeof lstatSync>; | ||
| try { | ||
| stats = lstatSync(path); | ||
| } catch (error) { | ||
| return error instanceof Error ? error.message : String(error); | ||
| } | ||
| if (stats.isSymbolicLink()) return "is a symbolic link"; | ||
| if (kind === "file" ? !stats.isFile() : !stats.isDirectory()) return `not a regular ${kind}`; | ||
| if (process.platform === "win32") return null; | ||
| const uid = process.getuid?.(); | ||
| if (uid !== undefined && stats.uid !== uid) return "owned by another user"; | ||
| if ((stats.mode & 0o022) !== 0) return "writable by group or others (chmod go-w)"; | ||
| return null; | ||
| } | ||
|
|
||
| /** Null when the file is safe to execute, otherwise the reason it is refused. */ | ||
| export function pluginFileTrustError(file: string): string | null { | ||
| return trustError(file, "file"); | ||
| } | ||
|
|
||
| /** | ||
| * A directory another user can write lets them add or swap plugin files, so its owner and | ||
| * mode are checked like each file's. | ||
| */ | ||
| export function pluginDirectoryTrustError(dir: string): string | null { | ||
| return trustError(dir, "directory"); | ||
| } | ||
|
|
||
| /** | ||
| * Every directory above the (resolved) plugin directory, up to `/`, must be owned by the user | ||
| * or root and not writable by group or others unless it is sticky (like `/tmp`), where others | ||
| * cannot rename or replace entries they do not own. With no writable component on the path, no | ||
| * other user can swap what the loader checked for something else before it is imported | ||
| * (OpenSSH StrictModes applies the same rule). POSIX only. | ||
| */ | ||
| export function pluginAncestorsTrustError(realDir: string): string | null { | ||
| if (process.platform === "win32") return null; | ||
| const uid = process.getuid?.(); | ||
| let current = dirname(realDir); | ||
| for (;;) { | ||
| let stats: ReturnType<typeof lstatSync>; | ||
| try { | ||
| stats = lstatSync(current); | ||
| } catch (error) { | ||
| return `${current}: ${error instanceof Error ? error.message : String(error)}`; | ||
| } | ||
| if (uid !== undefined && stats.uid !== uid && stats.uid !== 0) return `${current} is owned by another user`; | ||
| if ((stats.mode & 0o022) !== 0 && (stats.mode & 0o1000) === 0) { | ||
| return `${current} is writable by group or others`; | ||
| } | ||
| const parent = dirname(current); | ||
| if (parent === current) return null; | ||
| current = parent; | ||
| } | ||
| } | ||
|
|
||
| function isPlugin(value: unknown): value is OcxPlugin { | ||
| return typeof value === "object" && value !== null && typeof (value as OcxPlugin).setup === "function"; | ||
| } | ||
|
|
||
| async function withTimeout<T>(work: Promise<T>, ms: number, label: string): Promise<T> { | ||
| let timer: ReturnType<typeof setTimeout> | undefined; | ||
| const timeout = new Promise<never>((_, reject) => { | ||
| timer = setTimeout(() => reject(new Error(`${label} did not finish within ${ms}ms`)), ms); | ||
| }); | ||
| try { | ||
| return await Promise.race([work, timeout]); | ||
| } finally { | ||
| clearTimeout(timer); | ||
| } | ||
| } | ||
|
|
||
| export interface LoadOcxPluginsOptions { | ||
| /** Deadline for a setup that yields; see the module comment. */ | ||
| setupTimeoutMs?: number; | ||
| } | ||
|
|
||
| export async function loadOcxPlugins( | ||
| dir = pluginDirectory(), | ||
| options: LoadOcxPluginsOptions = {}, | ||
| ): Promise<PluginLoadResult[]> { | ||
| if (process.env["OCX_PLUGINS"] === "0") return []; | ||
| let files: string[]; | ||
| try { | ||
| files = listPluginFiles(dir); | ||
| } catch (error) { | ||
| return [{ file: dir, name: "plugins directory", loaded: false, error: error instanceof Error ? error.message : String(error) }]; | ||
| } | ||
| if (files.length === 0) return []; | ||
| const dirRefused = pluginDirectoryTrustError(dir); | ||
| if (dirRefused) return [{ file: dir, name: "plugins directory", loaded: false, error: `refused: ${dirRefused}` }]; | ||
| // Check and import through the resolved path, so both refer to the same components. | ||
| let realDir: string; | ||
| try { | ||
| realDir = realpathSync(dir); | ||
| } catch (error) { | ||
| return [{ file: dir, name: "plugins directory", loaded: false, error: error instanceof Error ? error.message : String(error) }]; | ||
| } | ||
| const ancestorRefused = pluginAncestorsTrustError(realDir); | ||
| if (ancestorRefused) return [{ file: dir, name: "plugins directory", loaded: false, error: `refused: ${ancestorRefused}` }]; | ||
| const results: PluginLoadResult[] = []; | ||
| for (const file of files.map(listed => join(realDir, basename(listed)))) { | ||
| const fallbackName = basename(file).replace(/\.(ts|js|mjs)$/, ""); | ||
| const refused = pluginFileTrustError(file); | ||
| if (refused) { | ||
| results.push({ file, name: fallbackName, loaded: false, error: `refused: ${refused}` }); | ||
| continue; | ||
| } | ||
| const unregister: Array<() => void> = []; | ||
| // Closed when setup fails or times out: a setup that resumes later must not register. | ||
| let active = true; | ||
| let shutdownCount = 0; | ||
| const whileActive = (name: string, register: () => () => void): void => { | ||
| if (!active) { | ||
| console.error(`[plugin:${name}] registration after a failed setup was ignored`); | ||
| return; | ||
| } | ||
| unregister.push(register()); | ||
| }; | ||
| try { | ||
| const module = await import(pathToFileURL(file).href) as { default?: unknown; plugin?: unknown }; | ||
| const plugin = module.default ?? module.plugin; | ||
| if (!isPlugin(plugin)) throw new Error("default export must be { name?, setup(context) }"); | ||
| const name = typeof plugin.name === "string" && plugin.name.trim() ? plugin.name.trim() : fallbackName; | ||
| const context: OcxPluginContext = { | ||
| name, | ||
| configDir: getConfigDir(), | ||
| pluginDir: dir, | ||
| log: message => console.log(`[plugin:${name}] ${message}`), | ||
| registerUpstreamRewriter: rewrite => whileActive(name, () => registerUpstreamRewriter(name, rewrite)), | ||
| // Keyed by file and registration, not name: two plugins may share a display name, and | ||
| // one plugin may register several teardowns. | ||
| onShutdown: teardown => whileActive(name, () => registerOptionalShutdownHook(`plugin:${file}#${++shutdownCount}`, teardown)), | ||
| }; | ||
| const timeoutMs = options.setupTimeoutMs ?? SETUP_TIMEOUT_MS; | ||
| await withTimeout(Promise.resolve(plugin.setup(context)), timeoutMs, `plugin "${name}" setup`); | ||
| results.push({ file, name, loaded: true }); | ||
| } catch (error) { | ||
| // A half-initialised plugin must not leave hooks behind, now or later. | ||
| active = false; | ||
| for (const undo of unregister) undo(); | ||
| results.push({ | ||
| file, | ||
| name: fallbackName, | ||
| loaded: false, | ||
| error: error instanceof Error ? error.message : String(error), | ||
| }); | ||
| } | ||
| } | ||
| return results; | ||
| } | ||
|
|
||
| /** `ocx start` entry: load and print one line per plugin. Never throws. */ | ||
| export async function loadAndReportOcxPlugins(): Promise<void> { | ||
| try { | ||
| for (const result of await loadOcxPlugins()) { | ||
| if (result.loaded) console.log(`🔌 Plugin loaded: ${result.name}`); | ||
| else console.error(`⚠️ Plugin ${result.name} skipped: ${result.error}`); | ||
| } | ||
| } catch (error) { | ||
| console.error(`⚠️ Plugin loading failed: ${error instanceof Error ? error.message : String(error)}`); | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.