Skip to content

feat: entrypoints — invoke a target in your world through the driver's ordinary run - #516

Open
raphaelvigee wants to merge 50 commits into
masterfrom
raphaelvigee/entrypoint-driver
Open

raphaelvigee wants to merge 50 commits into
masterfrom
raphaelvigee/entrypoint-driver

Conversation

@raphaelvigee

@raphaelvigee raphaelvigee commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Implements the entrypoints spec (draft 3, with board round-1 resolutions R1–R24 and round-2 decisions R25–R27, and R29 replacing R1): https://claude.ai/artifact/LFRqMFJj2yZZrsQT8eGp1b

An entrypoint target is run in your world instead of being built. heph run //scripts:migrate -- --dry-run builds and caches what the script needs, then runs it uncached, in your cwd, with your environment and your terminal, and exits with its status.

This PR is a rework of its own first version. The prepare hook (EntrypointDriver, PreparedInvocation, and the spawn in the binary) is gone. An entrypoint now runs through the driver's ordinary Driver::run/run_shell, with a few generic additions to the contract.

What changed

Contract (crates/plugin, crates/core/src/process.rs)

  • TargetDef.entrypoint: Option<Entrypoint { accepts_args, workdir }>.
  • RunRequest.options: RunOptions { cwd, base_env, args, stdio }, where stdio is RunStdio::Inherit(Foreground { signals, dispositions, nofile, blocked }).
  • ProcessFailed.exit: Option<Exit { Code, Signal }>.
  • The new types are #[non_exhaustive] and have constructors.

ABI 0.17.1 (additive; MIN_PLUGIN_ABI stays 0.17.0)

  • New fields: TargetDef.entrypoint = 11, ManagedRunRequest.options = 14 (nofile = 6 and blocked_signals = 7 inside it), RunInFrame.signal = 4, and RunOutFrame.exit = 5. exit sits outside the oneof, and error stays the terminal frame.
  • CreateConfig.host_capabilities tells a plugin whether the host supports entrypoints; on a host that doesn't, the SDK refuses to declare one.
  • The host's run request stream is now live, so signals reach the plugin during the run. N11 rule: the host closes that stream only after the terminal frame.

exec/bash

  • Options: entrypoint = True, args = True, and workdir = "caller"|"root"|"sandbox".
  • parse refuses incompatible fields (the spec's refusal table).
  • run_inner honours RunOptions generically: cwd, caller env as the bottom layer, args appended ("$@" in bash), and an inherited-stdio arm.
  • --shell has an args mode: positional parameters, run/xrun override them, and there is no runN.

execrunner foreground spawn

  • Inherited fds; the tool stays in heph's process group (the interim model).
  • Before exec, the child gets back the caller's dispositions, signal mask and RLIMIT_NOFILE.
  • A relay forwards TERM/HUP/INT/QUIT/USR1/USR2/WINCH. It skips signals the terminal already delivered: INT/QUIT/HUP/WINCH while heph is in the foreground (R2).
  • The child's exit is observed with waitid(WNOWAIT); the relay then stops, the pid is untracked and only then reaped, so no signal reaches a reaped pid. The supervisor registers the pid.

Engine

  • Engine::invoke and run_in_sandbox give execute and invoke one shared sandbox block (SandboxLease).
  • Each invocation gets a unique sandbox at <home>/sandbox/<pkg>/__invoke/<name>/<uuid>. It is created exclusively, guarded by a flocked .owner, always removed, and crash leftovers are swept by the next invocation and by heph gc.
  • The closure is laid out as for a build (R29, replacing R1): transitive tools and read_only_deps are linked from the shared stage and held for the run; ordinary deps are copies. read_only_deps is accepted on an entrypoint. Accepted: a tool writing through a linked input changes the shared stage entry.
  • R30 (user-approved per-platform difference): per-file staged tools are clonefile clones on macOS and hardlinks on Linux; each stage entry caches its file list (<hash>.files).
  • R24, measured on darwin/arm64 (warm, median, vs the previous design 69b7ed2d): bare entrypoint 15.2 vs 14.1 ms; 2,000 siblings 21.8 vs 20.6 ms; ~100-file transitive tool 40.6 vs 21.9 ms; 16 concurrent with that tool 575 vs 89 ms; an SDK-sized read_only_deps linked in 40 ms. A cold 1,000-package Go build is unchanged against master. The remaining tool-closure gap comes from laying files out per invocation, which R29 requires; it is accepted. The measurement also found a sweep/claim race (~1 in 120 concurrent invocations exited 125), fixed in d31ba6dc with a stress test that failed 3/3 before the fix.
  • Other engine rules:
    • The entrypoint attribute is kept across apply_transitive (R13).
    • Refusals (R14): an entrypoint can't be another target's input, and must be the only target its selection resolves to.
    • Cache is forced off, and pending sqlite writes are drained before the spawn.
  • Stage holds: stage_and_link takes a shared hold and returns it through sandbox_guards; clear_stage honours it (try-lock, remove .ready first). This also fixes an existing race between gc and a build.

Binary

  • At the top of main heph records the state it inherited: dispositions (SIGPIPE via a pre-main constructor), signal mask and NOFILE. The self-upgrade execv passes that state on to the pinned binary.
  • A refcounted transparency hold is taken at the handover: after materialization, right before the spawn.
  • The relay is installed at the handover. Once the last tool exits, Ctrl-C/TERM/HUP cancel any remaining uploads silently (R27).
  • heph run exit codes:
    • the tool's own code;
    • the tool's signal, re-raised;
    • 125: heph failed after the target resolved;
    • 130: Ctrl-C before the tool started;
    • 1: heph failed before the target resolved.
  • Trailing args are last = true.
  • New docs/ENTRYPOINTS.md.

Also fixed along the way

  • macOS fork race: a libnotify fork race made a fork die before exec, so a missing program "spawned". hproc::fork_safety::prime() is now called at the top of main and at the spawn sites.
  • Supervisor sidecar: it now ignores signals sent to heph's group. Before, a terminal Ctrl-C killed it, and a later kill -9 of heph left the tool unreaped.
  • Concurrent cold invocations: they no longer serialize behind a running tool's dependency read locks.
    • Reason: you decided (R25) that dependency locks stay held for the whole run.
    • Fix: a contended miss waits on the gateway, re-probes under a read lock and shares the hit (contend_for_write, jittered backoff).
    • Diagnostics (R26): the wait-for-lock notice now fires in every wait arm.

Release notes

  • heph run on an entrypoint = True target runs it in your terminal, directory and environment. Exit status: the tool's code, or heph dies of the tool's signal; 125 for heph's own failure after the target resolved (dep, refused argument, closure, credentials, spawn, or a driver reporting an impossible exit status); 130 for Ctrl-C before the tool started; 1 before resolution.
  • heph run -- //x no longer names a target: everything after -- is the tool's (last = true). Write heph run //x -- args. heph run //x word is refused, with a message naming the -- spelling.
  • Signals:
    • Before the tool starts, heph behaves as for any run.
    • While the tool runs, heph forwards TERM, HUP, INT, QUIT, USR1, USR2 and WINCH sent to its pid, and acts on none of them.
    • After the tool exits, Ctrl-C/TERM/HUP cancel any remaining uploads silently.
    • Limits (X5): a kill -INT/-QUIT/-HUP <heph pid> sent while heph is the terminal's foreground is dropped, and a signal sent to the whole group may reach the tool twice.
  • The process supervisor sidecar now ignores signals sent to heph's process group.
  • Every command started through a self-upgraded (pinned) binary restores the caller's signal dispositions, mask and open-file limit before the execv.
  • A BUILD file using entrypoint/args/workdir on exec/bash needs a heph with entrypoints; an older heph fails to parse it. An entrypoint named __target_* is refused.
  • Plugin SDK: source-incompatible, wire-compatible, ABI 0.17.1. RunRequest, RunOptions, Foreground, Entrypoint and RemoteProcessFailed are #[non_exhaustive] with constructors. ProcessFailed gains exit. Use ForegroundCommand::with_foreground(&fg) to apply the caller's state.
  • heph inspect def adds "runs_as": "<driver>, entrypoint (uncached)" for an entrypoint. It is a human string; programs key on target_def.entrypoint.

Deviations from the spec

  • cdylib_entrypoint_runs has two layers.
    • bin-e2e, real dlopen (crates/bin-e2e/tests/cdylib_entrypoint.rs): it loads a new test-only cdylib, crates/plugin-entrypoint-test-cdylib, into the shipped binary through a manifest. Under a PTY it asserts the caller's cwd and env, HEPH_ROOT, a tty, args a/b c, exit 42 passing through, and SIGTERM to heph's pid reaching the tool exactly once.
    • Build and release wiring: the e2e script and the CI build job build and stage the cdylib as test-entrypoint-plugin_*. It is never published: tests/ci_gate.rs test_fixtures_are_never_published guards that.
    • In-process seam test in crates/e2e: kept, because it cheaply covers more rows. The cdylib leg of entrypoint_survives_apply_transitive runs only there.
  • Where the transparency hold is taken. It is taken at a host-only Handover hook after materialization, not "before the fork", so Ctrl-C still cancels while the closure materializes. The TUI pauses with PauseFor::Output, not Input.
  • R19 is wider than its regex. It also refuses a bare $SRC/$TOOL/$OUT, and $SRC_x even inside an exec sh -c element, because heph can't tell expansion from literal text. The message says to use driver = "bash" or ${src://…}.
  • Workdir is exhaustive, not #[non_exhaustive]: the host matches on it. A cdylib failure surfaces as RemoteProcessFailed { message, exit }.
  • Open 8 (FUSE): an entrypoint always uses the OS runner. fuse.enabled is ignored for it, and the docs say so.
  • Open 9: invocation sandboxes take no generation claim, so that map doesn't grow.
  • The coreutils toolbox version now folds into an entrypoint's def hash.
  • Open 6: heph run has no --json, so there is nothing to shape.
  • Open 7: a --shell session forwards only TERM and HUP, to its pgid.
  • X6 is gone: an inherited SIGPIPE ignore is recorded before std's runtime resets it.

Not done / follow-ups

  • The self-upgrade exec_into row now exists: self_upgraded_heph_keeps_callers_open_file_limit_and_mask (bin-e2e). It runs offline: a copy of the binary under test is pre-staged as the pinned version in the test's HOME. Not shown red against a binary without the restore.
  • A cancel landing inside materialization. No hook can time a test there. The cancel check right before the spawn catches it.
  • perf-measurement (R24) was NOT MEASURED locally: the disk was full. The PR carries ci/perf-test, so CI's Perf job (baseline vs candidate heph-bench) is the measurement. It covers the general build paths (D1, N3), not R24's entrypoint-specific setups.
  • signal_after_the_tool_exits_cancels_the_uploads (bin-e2e) failed on its first CI run, as did the SIGQUIT row of entrypoint_signal_death_is_reraised on Linux. Both are fixed:
    • b3f9ed4: cancel_all_requests now reaches work that outlives the root request state; the upload ran on a child state the registry no longer found.
    • ab245af: a re-raised signal now dumps no core on Linux. RLIMIT_CORE 0 doesn't stop a piped core_pattern, so heph also calls prctl(PR_SET_DUMPABLE, 0).
    • The entrypoint bin-e2e file passes locally on macOS (16/16). Linux CI is the check for the core fix.

Board

  • Round 1 (design): recorded in the spec.
  • Round 2 (review, c97a6cd), then confirmations at 1fb2cf1 and c6d614e:
    • product-vision: SHIP WITH CHANGES. Its major (the sh -c wording of the refusal message) is fixed in 1fb2cf1.
    • feature-quality: BLOCKED (no test for the NOFILE restore) → PASS WITH FOLLOW-UPS at 1fb2cf1.
    • code-quality: PASS WITH NITS. The major (TERM/HUP swallowed before the tool) is fixed in 9e2708c. The post-tool signal minor is fixed per R27 in baf1f3a.
    • hermeticity: HERMETIC WITH EXEMPTIONS. The major (the managed layer linking from the stage for a non-exec entrypoint) is fixed in dcda8d5.
    • compatibility: COMPATIBLE. The major (TERM/HUP) is fixed in 9e2708c.
    • perf-measurement: NOT MEASURED (disk).
  • Exemptions:
    • X5: accepted by code-quality, product-vision, hermeticity and compatibility.
    • X6: removed.

Tests

lint passes on HEAD. Spot runs passed for these crates: core, proc, execrunner, engine, driver-support, driver-bridge, plugin, plugin-abi (convert), plugin-sdk, plugin-stabby, plugin-exec, selfupdate, builtins, plugin-oci, and the root crate's lib, bins and supervisor tests.

e2e and bin-e2e files that passed:

  • e2e --test entrypoint (25 tests), plus engine_core, clean, credential, scratch, tool_env, deferred, group, query, oci_image, http_fetch and plugin_fn_declarations.
  • bin-e2e entrypoint, entrypoint_pty and shell_pty, at 1fb2cf1. The one test added after that hasn't run yet; see "Not done".

🤖 Generated with Claude Code

raphaelvigee and others added 30 commits October 28, 2026 18:52
…t split

D47 of the tool-shims spec: a golden over exec's parse() and
apply_transitive() output for a fully populated spec (deps, hash_deps,
runtime_deps, tools, env, out, support_files, runner, pass_env, and a
transitive closure with tools, deps and hashed/unhashed env), for exec,
exec with coreutils, and bash. The exec.entrypoint driver splits parse and
apply_transitive into shared helpers; this pins the hashes and input lists
first, so that split cannot move an exec def hash unseen.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first slice of the tool-shims spec: an `exec.entrypoint` target runs in
the caller's world — their cwd (or the workspace root), their environment,
their stdio — instead of a sandbox, and `heph run //t:x -- args` runs it.

Driver (crates/plugin-exec, D15, D19, D26, D8): its own EntrypointDef and
def hash; exec's dependency vocabulary (deps, hash_deps, runtime_deps, tools,
env, transitive) through parse/apply_transitive helpers shared with exec by
name, not a branch on the driver name (the D47 golden pins exec's output
across the split). Parse refuses out, cache, codegen, support_files,
scratch, pass_env, runtime_pass_env, runtime_env, credentials (D28), runner
("not supported yet": needs base_env, ABI 0.18.0), workdir other than
"caller"/"root" (D11), unknown fields and wrong types, each naming the field
and target. `args` on exec/bash is refused (D18). The shim label is not
checked at parse (D36).

Engine (D1, D30, D43, D46, I2, I6, I8): the Entrypoint capability is a
builtin-only PluginDriver variant. Engine::invoke resolves the spec, refuses
undeclared args before any dep builds, builds the closure, drains the sqlite
write-behind and pending remote uploads, resolves ${read://} through
resolve_deferred, stages every dep in <home>/stage and holds it, flattens
the tools into a held toolbin stage entry, and hands the driver the private
Invocation token to shape argv/env/cwd. No execute lock. Cache is forced off
from the capability in get_def_inner. An entrypoint is refused as an input
in expand_inputs (deps, runtime_deps, tools, hash_deps, runner, read://,
transitive closures), in multi-target selections, and by an execute
backstop for a direct result_addr.

Holds (I4, D27, proposed in the spec): a shared flock on each stage entry's
<hash>.lock, taken then re-checked (.ready, entry dir; FRWLock checks the
inode), re-materialized on a miss, deduped by path. clear_stage now deletes
.ready before the entry.

Binary: the binary, not the engine, spawns the prepared command, stdio
inherited, env built as OsString (no filter_long_env, D29), and exits with
the child's code or re-raises its signal with RLIMIT_CORE at 0. Process
model is a temporary stand-in for D2: the child stays in heph's process
group, system(3)-style (SIGINT/SIGQUIT ignored while waiting, reset with
SIGTTOU/SIGTTIN in the child).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
D13 of the tool-shims spec: `heph run --shell //t:tool -- a 'b c'` opens an
interactive bash in the tool's world — its env, PATH order (toolbin first)
and cwd — with the closure built and held while the shell lives; heph exits
with the shell's status.

It reuses exec's machinery: the same init.sh template in its own
`{% if entrypoint %}` branch (exec's render is pinned byte for byte by
exec_shell_init_is_byte_identical), and exec's host-PATH lookup for bash
(resolve_on_host_path, now shared with resolve_shell). The command line is
the entrypoint's resolved argv, each word quoted, plus the args; the caller's
args are bash's positional parameters (`bash --noprofile --rcfile <init> -i
-s -- <args>`). `run`/`xrun`/`run1`/`xrun1` take their own args, which
replace the originals for that call; a bare call always means the originals.
`show`/`showl` print the args %q-quoted. No $HEPH_CMD or $HEPH_ARGS.

Deviation from the spec, forced by bash: an interactive bash binds its
positional parameters only after reading its rcfile, so init.sh copies them
at top level (the sourced case) and again in a one-shot PROMPT_COMMAND
before the first prompt, which then restores the caller's.

init.sh is a per-invocation file under <home>/tmp, never the cwd, removed
with the invocation. Undeclared args are refused before anything builds, as
for a plain run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The `exec.entrypoint` driver and its `EntrypointDef` are gone. An
entrypoint is now `entrypoint = True` on an `exec` or `bash` target, with
`args` and `workdir` as exec spec fields and an
`Option<EntrypointMode { args, workdir }>` on exec's def.

- Exec's def hash and serialized def do not move without the flag: the
  mode is skipped by both Hash and serde when absent. The D47 golden is
  unmodified and green. With the flag the hash covers entrypoint, run,
  env, workdir and args; the coreutils version is not folded in (D24).
- With the flag, parse keeps the old refusals (out, cache, codegen,
  support_files, scratch, read_only_deps, pass_env, runtime_pass_env,
  runtime_env, credentials, runner, workdir other than caller/root), each
  naming the field and the target, and skips the workspace runner
  default. Without it, `args`/`workdir` are refused, pointing at
  `entrypoint = True`. An entrypoint's `env` takes `${read://}`.
- `bash(entrypoint = True)` runs its script through exec's bash
  invocation with the caller's args as positional parameters ("$@"),
  directly and under `--shell`.
- Engine: `TargetDef.entrypoint` (serde-skipped when false) replaces the
  driver capability for cache-off, the input refusal, the execute
  backstop, the selection refusal and `invoke`. The `EntrypointDriver`
  hook stays, no longer a `Driver` subtrait, registered with
  `PluginParts::with_managed_entrypoint_driver`; `invoke` refuses a
  flagged def whose driver has no hook (`EntrypointWithoutHook`).
- No ABI change: `target_def_from_pb` sets `entrypoint = false`, so a
  cdylib driver cannot declare an entrypoint.

Every `TargetDef` literal gains `entrypoint: false` (compiler-listed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An entrypoint has one command line, so its `--shell` offers `run`,
`xrun`, `show` and `showl` only; `run1`/`xrun1` and their help line are
gone. `run`/`xrun` keep the arg override. Exec's `init.sh` render is
unchanged (`exec_shell_init_is_byte_identical` passes unmodified), and
`entrypoint_shell_init_args_exact` drops its `run1` rows for one
asserting `run1`/`xrun1` are not defined.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a hand-written Default for TargetDef (unit raw_def, caching off) and use
struct-update syntax in test literals, dropping fields that equal the default.
Production literals keep an explicit entrypoint: false.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Production TargetDef literals (driver parse implementations and the wire
conversion) now use struct-update syntax too, dropping every field equal to
the Default impl. The only explicit entrypoint: false left is the Default impl.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Finish the ..Default::default() pass: engine TargetDef literals still carrying
empty inputs/outputs or cache off, the production exec literal, and exec's own
def struct, which now derives Default so its test literals state only what
they set.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rebase onto master brought #519, which made exec's `env` and
`runtime_env` `Deferred<String>` maps; the entrypoint hook still read them
as `String`s and plugin-exec no longer compiled. Keep the hook's behaviour
(raw text for the closure's `runtime_env`, resolved values for `env`)
with the new types. The hook itself goes away in the entrypoint rework.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e driver contract

The driver-contract half of the entrypoint rework: an entrypoint runs through
the driver's ordinary `run`, so the contract carries what that run needs.

- `TargetDef.entrypoint: Option<Entrypoint { accepts_args, workdir }>`
  replaces the bool; `Workdir` is `Caller`, `Root` or `Sandbox`. Skipped in
  `inspect def --json` when unset. Readers only switch to `is_some()`.
- `RunRequest.options: RunOptions { cwd, base_env, args, stdio }`, default for
  every build. `RunStdio::Inherit(Foreground { signals, dispositions })` carries
  the relay receiver and the dispositions the child execs with. `RunOptions`
  is not `Clone`, and its `Debug` prints `base_env` keys only.
- `RunRequest` is `#[non_exhaustive]`, built with `RunRequest::new` plus field
  assignment; `with_target` moves a request onto another def (the shell
  fallback), naming every field so none can be dropped there.
- `ProcessFailed.exit: Option<Exit>`, `Display` unchanged; exec fills it.
  `RemoteProcessFailed` and `process_exit()` read the status back whichever
  side of the plugin seam produced it.
- `Exit`, `Signal` and `Disposition` live in `hcore::process`, shared with
  `execrunner`, whose `ForegroundExit` is now an alias of `Exit`.

Behaviour is unchanged: nothing sends non-default options yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lugin ABI

ABI 0.17.1, an additive patch; MIN_PLUGIN_ABI stays 0.17.0.

- `TargetDef.entrypoint` (11): absent decodes to `None`, so a def from an
  older plugin is never an entrypoint; an unknown workdir is refused.
- `ManagedRunRequest.options` (14): `RunOptions` with cwd, env and args as
  bytes (non-UTF-8 survives), wrapper messages keeping unset apart from
  empty, and the stdio mode with its dispositions. `run_options_to_pb`
  refuses non-default options for a def that declared no entrypoint (N9),
  so the host never sends them to a plugin that would run them as a build.
- `RunInFrame.signal` (4): an OS-local signo, never persisted.
- `RunOutFrame.exit` (5), outside the oneof: `error` stays the one terminal
  failure frame, so an older host still prints the message.
- `CreateConfig.host_capabilities` (5): the `entrypoints` bit. Absent is no.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…by seam

Host (`StableRemoteManagedDriver::dispatch_run`):
- encodes `RunOptions` on the start frame, refusing them for a def that
  declared no entrypoint (N9, at the seam);
- the run's request stream is now live: a channel the host owns instead of
  the fixed `HostItemStream` Vec. The relay forwards each signal from
  `Foreground::signals` as `RunInFrame.signal`, and the sender is dropped
  only once the terminal frame has been read (N11);
- an `error` frame with an exit status becomes `RemoteProcessFailed`, whose
  message is the frame's, unchanged.

Guest (SDK):
- rebuilds `RunOptions`; an inherited run gets no output sinks, and a
  dedicated thread feeds the rest of the request stream's signal frames into
  the driver's receiver. The terminal frame never waits on it, and an early
  end (an older host) is "no signals", not a cancel;
- a failure's `ProcessFailed.exit` rides out on `RunOutFrame.exit`;
- `create_config_from_bytes` records the host's capabilities per plugin, and
  every def a guest `parse`/`apply_transitive` returns is refused if it
  declares an entrypoint the host does not advertise
  (`require_entrypoint_support` lets a driver check earlier).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…:Inherit

`spawn_foreground` runs a program in the caller's terminal: inherited fds
0-2, no setsid, heph's process group (the interim model; D2 comments at
each place the shared group matters). In the child, allocation-free,
INT/QUIT/TTOU/TTIN go back to SIG_DFL, the host's recorded dispositions
apply, and the recorded RLIMIT_NOFILE soft limit is restored.

`ForegroundChild::wait` selects over the run's signal channel and the
child's exit in one task: the exit is observed without reaping
(waitid WNOWAIT, Linux and macOS alike), then relaying stops, the pid is
untracked from the supervisor, and only then is the child reaped, so a
relayed signal never reaches a reaped pid. The skip decision for
terminal-generated signals (R2) is the pure `should_relay`, applied at
delivery, where the tool's pid is known.

Re-exported from hplugin_sdk::foreground for third-party drivers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l table

Parse (entrypoint_mode.rs): `entrypoint = True` takes `args` and
`workdir = "caller"|"root"|"sandbox"`, and refuses out, cache, codegen,
support_files, read_only_deps (R1), pass_env, runtime_pass_env,
runtime_env, runner ("not supported yet") and scratch outside sandbox
mode, each message naming the field, the target and what to do instead.
Credentials are accepted (E18). A non-bool flag is refused. An exec
entrypoint argv naming `$SRC_x`/`${TOOL_x}`/`$OUT` is refused pointing at
`${src://}` and `driver = "bash"` (R19). The def hash folds in
Some(EntrypointMode) and nothing when unset (golden unchanged); the
engine attribute carries Sandbox. An entrypoint's transitive tools are
copied, not linked from the shared stage (R1).

Run: `run_inner` honours RunOptions generically: cwd override (a missing
directory is named), base_env as the raw bottom env layer with exec's
layers on top and no search_path PATH (bin/ first, coreutils last), args
appended to argv or as bash's "$@" (with a $0 for `-c`, none for a
spilled script), E2BIG named. `RunStdio::Inherit` spawns through
execrunner's foreground spawn and relays the run's signals; a failure
carries `ProcessFailed.exit` and an empty log, never a sandbox handle
(R22). `--shell` renders init.sh's args mode whenever
`RunOptions.args.is_some()` (E11), and forwards only TERM/HUP to the
session's group (Open 7). Without args init.sh is byte-identical.

The prepare hook (entrypoint.rs) stays until the engine and binary stop
calling it (layer 3). The e2e caller-cwd test moves its `sh -c` script
to `driver = "bash"`, as R19 now requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sandbox_guards

Every stage entry a run links to is held from link until the run's guards
drop (N3, E14). `stage_and_link` materializes, takes a shared flock on the
entry's `<hash>.lock`, re-checks `.ready` under it (bounded retries when a gc
lands in the window), then links, and returns the hold. The managed OS
runner hands the holds out through `RunResponse.sandbox_guards`, which the
engine drops after the cache write; on any failure they drop with the frame.
`clear_stage` already tries the exclusive lock, skips a held entry, and
removes `.ready` before the tree.

The invocation-only `StageHolds`/toolbin machinery of the prepare hook goes:
an invocation copies its closure (R1) and stages nothing of its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`execute`'s sandbox block becomes `Engine::run_in_sandbox`, shared with
`invoke` (R12, E7) and parameterised by a `SandboxLease`: a build's
per-address directory (generation-checked stale removal; kept for
diagnostics on failure) or an invocation's own one (created empty and
exclusively; removed on every exit). It returns heph's failure and the
driver's run apart, and refuses non-default run options for a def that is
not an entrypoint (N9).

Invocation sandboxes (R11, N10, E20) live at
`<home>/sandbox/<pkg>/__invoke/<name>/<uuid>`: an exclusive leaf create
retried on AlreadyExists only, an `.owner` file locked before it gets its
name, the lock held until the tree is gone. A crash leftover — `.owner`
present and free — is swept by the next invocation of the target and by
`heph gc` (`GcStats.invoke_sandboxes_removed`), never through a symlink and
never into a build's `__target_*` sandbox. No generation claim is taken, so
the never-pruned claim map does not grow per invocation (Open 9).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t its readers

A request that missed on a target while another built it used to block on
the revision's write lock, which waits for every reader to drain. Once the
builder downgraded, its consumers' riding reads kept the lock — for an
entrypoint, for as long as the tool runs, since its closure carries them —
so the loser waited for the winner's tool to exit instead of sharing the
build (N10).

The miss path now tries the write lock, and when it is taken waits only for
the target's gateway (free once any build of it ends), re-probes under a
read and shares a hit; a miss still contended was a passing prober, so it
backs off (5–100ms, cancellable) and tries again.

`concurrent_invocations_run_in_parallel` timed out after 120s before this
change, with one tool running and the other fifteen requests parked in
`ResultLock::write`; it passes in under a second after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… run

`Engine::invoke` reads `accepts_args` from `TargetDef.entrypoint`, refuses
undeclared args before any dep builds, builds the closure, drains the cache
writes it queued, and runs the shared sandbox block with an invocation lease
and `RunOptions`: cwd from the workdir (caller's, the root, or none for the
sandbox), the caller's env with `HEPH_ROOT` appended (R17), args only when
declared, and the caller's terminal with the relay receiver and dispositions
the caller hands in (R9: the engine installs no handler). It takes no execute
lock, no worker permit and no memoizer slot, writes no cache, and removes its
sandbox before returning, on every exit. One `info` event names the
invocation (R20).

It returns an `InvokeOutcome` (R4): `Exited(Exit)` when the tool ran — read
from the driver run's own `ProcessFailed.exit`, never a dep's — or
`Failed(err)` for a heph-side failure after resolution; `Err` stays a failure
before the target resolved.

The entrypoint attribute is the parse's (R13): restored when a driver's
`apply_transitive` drops it, refused when it names a different one. The FUSE
bridge always runs an entrypoint on the OS runner (Open 8).

The e2e suite moves to the run path and gains the spec's rows: the refusal
table, workdir modes with scratch, 16 concurrent invocations over two engines
with gc running, the sandbox removed on every exit, copies not links,
credentials, and the transitive tamper rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR #516's parallel universe goes, now that an entrypoint runs through its
driver's ordinary `run`: `hplugin::driver::entrypoint` (`EntrypointDriver`,
`Invocation`, `PrepareRequest`, `PreparedCommand`, `StagedInput`, `Caller`),
`PluginDriver::ManagedWithEntrypoint` and its registration, exec's prepare
hook, and execrunner's `run_foreground`/`ForegroundExit`. exec and bash
register as plain managed drivers again.

`heph run` calls `Engine::invoke` inside the app, the TUI paused for input
for the run, and exits as the tool did. The signal relay, the recorded
dispositions, the suppression guard and exit 125 are still to come
(`TODO(entrypoint-L4)`).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ches exec on macOS

A child forked while another thread of a fresh process was initializing
libSystem's libnotify globals died in the atfork child handler
(_os_once_gate_corruption_abort, SIGKILL) before exec. std then reads EOF
on its exec-error pipe and reports a successful spawn: a missing program
"spawned", a real one is "killed by signal 9". This was the execrunner
foreground_spawn_missing_program_is_not_found flake (~1 in 3 runs of the
lib tests, racing foreground_spawn_closed_channel_is_not_a_cancel).

hproc::fork_safety::prime() initializes the globals once before any fork
that goes through pre_exec (proc_exec on macOS, the foreground spawn).
Regression test first_concurrent_foreground_spawns_reach_exec races two
first spawns in 30 fresh processes: red on attempt 1 without the prime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pgrade

hcore::process::record_inherited() records the dispositions of every signal
heph touches (INT, QUIT, HUP, TERM, PIPE, TTOU, TTIN, USR1, USR2, WINCH) and
the open-file soft limit, before main changes any of them (R8). PIPE is
recorded as default (X6: Rust ignores it before main). The foreground spawn
reads the NOFILE record from there instead of its own static.

The self-upgrade's execv puts the recorded state back first
(restore_inherited_for_exec, in a pre_exec), so the pinned binary records
the caller's dispositions and limit rather than the TTOU/TTIN ignore and the
raised limit the first binary applied.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replaces the single suppression bool for entrypoints (R9). While any
hcore::shutdown::TransparencyGuard is alive the shutdown trigger drops every
press, its consumer drops presses queued before the hold, and the SIGQUIT
dump, the per-thread SIGUSR1 backtrace and the SIGUSR2 profile dump stand
down. Refcounted, so two invocations never end each other's hold.

Foreground gains a host-only Handover hook, fired once right before the
driver that spawns the program is called: by the managed layer after the
inputs are materialized (host side, so it covers cdylib drivers), or by the
engine for a plain driver (engine Driver.managed). Deps building and the
closure materializing therefore stay cancellable with Ctrl-C.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… it did

- The relay's listener (bootstrap::spawn_signal_relay) forwards TERM, HUP,
  INT, QUIT, USR1, USR2 and WINCH into InvokeRequest::with_signals,
  unfiltered: the driver applies the terminal skip (R2). heph no longer dies
  of them while the tool runs.
- The recorded dispositions go to with_dispositions; the handover takes the
  transparency hold until exit (R9). The TUI pauses with PauseFor::Output so
  a Ctrl-C while the closure materializes still cancels; the hold owns it
  from the handover on.
- Exit mapping (R4): the tool's code as is, its signal re-raised, 125 with
  the box for a heph-side failure after resolution, 130 for a Ctrl-C before
  the tool started, 1 before resolution (run::InvokeFailed).
- R18: an entrypoint reached by a selection names the selection's size once
  the walk finished (keep-going); `heph run -- a` and `heph run //x word`
  are refused with the `--` spelling to use.
- R20: the TUI header says "Invoking"; inspect def adds
  "runs_as": "exec, entrypoint (uncached)" for an entrypoint only.
- Docs: docs/ENTRYPOINTS.md and the run --help text (exit codes, signals,
  --shell + show, $WORKSPACE_ROOT vs $HEPH_ROOT).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sidecar shares heph's process group, so the terminal's Ctrl-C, Ctrl-\,
Ctrl-Z and hangup, and any kill -<pgid>, reached it too, and the default
action killed it. With an entrypoint holding the terminal the tool keeps
running after a Ctrl-C (it is the tool's), and a later kill -9 of heph then
had no supervisor left to reap it. It now ignores INT, QUIT, HUP, TERM,
TSTP, TTIN, TTOU, USR1 and USR2; it still ends when heph's end of its
socket closes.

bin-e2e supervisor_survives_ctrl_c_and_reaps_after_kill_9 (entrypoint_pty)
times out waiting for the reap without this, passes with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A shell runs on a PTY of its own and ignores RunOptions.stdio: the wrapper's
streams are how the terminal reaches that PTY. The shared sandbox block
dropped them for every Inherit run, so heph run --shell on an entrypoint
showed nothing and read nothing. They are now passed on to a shell run, and
the invocation's wrapper hands them in (TtyReader, as a build's --shell
does). The wrapper is also installed without a TUI, so the stall notice is
suppressed while any tool runs.

The heph-side failure path wraps finalize! in its own block: its early ?
returned the bare error, which then exited 1 instead of 125.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
entrypoint.rs: entrypoint_keeps_inherited_dispositions (plain caller vs
trap '' INT TTOU TTIN), entrypoint_signal_to_heph_is_forwarded (TERM and USR1
once each, a TERM that kills the tool leaves none), error_frame_without_exit_
exits_125_with_box (missing program, failing dep, refused arg -> 125; typo ->
1), sigkilled_heph_tool_reaped_and_sandbox_swept, ctrl_c_during_dep_build_
cancels (130, tool never ran).

entrypoint_pty.rs (vt100 cells via common::pty): entrypoint_tui_pauses_for_tool
("Invoking" label, tty on fds 0-2, cursor restored), entrypoint_ctrl_c_goes_
to_tool (two presses each delivered once, USR1 relayed once, heph silent,
exit 7 not 130), supervisor_survives_ctrl_c_and_reaps_after_kill_9.

shell_pty.rs entrypoint_shell_pty: the init file lives in the invocation
sandbox now; adds the show row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cdylib_entrypoint_runs: a third-party managed driver exported with
named_managed_driver and decoded by the host loader's decode_drivers, with
the host capability bit recorded from encoded CreateConfig bytes as a
cdylib's create entry does. Its parse declares the entrypoint, its run
spawns through hplugin_sdk::foreground honouring RunOptions. Engine::invoke
drives it: caller cwd, caller env + HEPH_ROOT, args byte for byte, exit code
as the outcome, and a SIGTERM pushed into the request's signal channel
reaching the tool's trap over the RunInFrame.signal relay.

entrypoint_survives_apply_transitive now runs every row in-process and
behind the seam (R13's cdylib leg).

In-process rather than bin-e2e with a test cdylib, following
plugin_fn_declarations.rs: what an entrypoint adds rides the run's protobuf
frames, so dlopen has nothing entrypoint-specific to reject.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… invoke start

Before the tool is about to be spawned heph is an ordinary run again: TERM
and HUP end it, Ctrl-C cancels, SIGQUIT dumps diagnostics and SIGUSR2 a
profile. The relay's channel still exists from the start so the request
carries its receiver, but its listeners are installed by the handover, so
nothing a user sent to a slow dep build is queued for a tool that does not
exist yet (a QUIT to inspect it no longer kills the tool later).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on Foreground

The recorded RLIMIT_NOFILE soft limit was restored only from a static in
the host's execrunner, so a plugin driver's foreground spawn (whose copy of
that static is never set) kept heph's raised limit (D5). It now rides the
run as Foreground.nofile and an additive RunOptions.nofile on the wire
(still ABI 0.17.1, unreleased), and ForegroundCommand::with_nofile sets it.

The caller's signal mask is recorded with the dispositions and restored the
same way (Foreground.blocked, RunOptions.blocked_signals): std clears the
mask in every child before its pre_exec, so without it a tool started under
a blocked signal ran with none blocked. The self-upgrade's exec restores it
too. A cancel seen after the runner's prepare now spawns nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runner agent, telemetry and fsutil fork too, and never called
fork_safety::prime(); priming once while the process is single-threaded
covers every fork the binary makes. The two spawn sites keep their own
call for processes that never run heph's main (tests, embedders).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
raphaelvigee and others added 8 commits October 29, 2026 01:48
R1 was enforced only by exec's apply_transitive dropping READ_ONLY; the
managed OS runner linked any read-only input from the stage regardless of
the def, so another driver's entrypoint could write through a link into the
shared stage entry. run_inner now treats the stage as absent for a def with
`entrypoint` set: every input is a writable copy and nothing is held.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…giene

- Drain only the local cache's queued writes before the tool starts, and
  cancellably (biased to the cancel): remote uploads run on beside the tool
  and are drained at exit like any command's, instead of delaying the spawn
  by up to their timeout; a Ctrl-C while a write is stuck ends the run
  before the handover.
- Refuse an entrypoint named `__target_*`: its invocations would land in
  the build sandbox of `//<pkg>/__invoke:<x>`.
- Remove the lock file last, so a teardown that fails partway stays
  sweepable; sweep an unlocked `.owner-pending`-only dir (a crash between
  the lock and the rename), the creator retrying another name if a sweeper
  locks its fresh one first; never walk into a live invocation's tree.
- A driver-reported exit status no failed process can have (code 0 or
  outside 1-255, a stop/ignore-class signal) is heph's failure (125, with
  the driver's message), not a status heph tries to exit with.

Tests: cancel_before_the_handover_spawns_nothing, abandoned_invocation_is_reclaimed,
entrypoint_name_with_build_sandbox_prefix_is_refused, failure_status_table, and
new rows in stale_invoke_sandbox_swept / sweep_all_finds_every_package.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
std ignores SIGPIPE before main, so the recorded disposition was always
Default (accepted limit X6). A pre-main constructor (.init_array /
__mod_init_func) now notes it before lang_start; the tool and the
self-upgraded binary get it back. X6 is no longer a limit.

bin-e2e: the PIPE rows in entrypoint_keeps_inherited_dispositions,
entrypoint_keeps_callers_open_file_limit_and_mask, term_or_hup_during_dep_build_ends_heph,
quit_during_dep_build_does_not_reach_the_tool, entrypoint_kill_under_a_terminal (TERM
once, INT dropped under X5, group TERM at least once); every non-PTY wait is
bounded and kills heph on timeout (common::wait_output, Workspace::run).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A Ctrl-C accepted before the handover cancels even when its consumer
  reads it after: presses are gated at trigger time only.
- diag's per-thread dump installs its SIGUSR1 handler once and never over
  someone else's (the relay's listener).
- ForegroundChild's drop kills, untracks, then reaps, as wait does.
- RunOptions::is_default destructures, so a new field cannot be missed (N9).
- RemoteProcessFailed is #[non_exhaustive] with a constructor.
- Driver contract docs: an entrypoint's run may be one of several in
  flight for one addr, each in its own sandbox_dir (N10); when the handover
  fires relative to run.
- The shell relay's killpg comment states its residual pgid-reuse window.
- first_concurrent_foreground_spawns_reach_exec loops only on macOS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he loop

The user's decision on D1: an entrypoint's dep result locks stay held for
the whole run, so the contended-miss loop (try_write; else wait on the
gateway, re-probe under a read, share a hit; else back off) stays.

- The whole contended wait is one acquire_with_notice scope: a target held
  by another process or request gets one ResultLockWaitStart (with the
  holder where known) after RESULT_LOCK_NOTICE, whichever arm it waits in —
  the backoff included, which emitted nothing — and one ResultLockWaitEnd.
- After a miss under the gateway, try the write lock before backing off;
  the backoff is jittered so racing waiters do not retry in lockstep.
- The first try_write carries 'acquiring result lock for <addr>'.
- The loop is a generic contend_for_write so each arm is unit-tested:
  winner failed -> wins try_write, late hit shared, backoff until free,
  cancel inside the backoff sleep, and the notice in the gateway, probe and
  backoff arms (silent under the threshold). e2e:
  dep_held_by_another_engine_announces_the_wait.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…X5, NOFILE

- The R19 refusal message and ENTRYPOINTS.md no longer claim exec 'has no
  shell' as the reason: the variable is refused even inside sh -c because
  heph cannot tell an intended expansion from literal text; bash (quoting a
  literal '$OUT') or ${src://...} is the way.
- Before the tool starts heph is an ordinary run (TERM/HUP end it, QUIT
  dumps), nothing is queued for the tool; materializing is silent while
  the TUI is paused but still cancellable.
- X5 (kill -INT/-QUIT/-HUP of heph's pid while it owns the terminal is
  dropped) and the group-signal double delivery are documented; X6 is gone
  (an inherited SIGPIPE ignore is kept), as is the builtin-only NOFILE note.
- FUSE is ignored for entrypoints; runs_as is for people, programs read
  target_def.entrypoint (now pinned by the inspect def test).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…R27)

Transparency was held, and the relay kept swallowing TERM/HUP, until heph
exited, so a post-tool drain of a stalled remote upload (up to
UPLOAD_DEADLINE) ignored everything but SIGKILL. Now the handover's
TransparencyGuard is handed to hcore::shutdown::tool_exited once invoke
returns: when no other tool runs, Ctrl-C, TERM and HUP cancel what heph is
still finishing (the relay's listeners call back into cancel_all_requests
instead of forwarding), silently — no message, no hard abort, no 130 — and
heph exits with the tool's status. The refcount still keeps a second
invocation's tool transparent. The relay arms all listeners before
spawning their tasks; the instant inside tokio's signal() is documented.

Tests: core tool_exited_ends_only_its_own_transparency; bin-e2e
signal_after_the_tool_exits_cancels_the_uploads (an S3 endpoint that accepts
and never answers; INT and TERM each end heph within 20s with exit 3).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…error context

- hcore::process::CallerState, implemented by Foreground, lets
  ForegroundCommand::with_foreground(&fg) apply dispositions, nofile and
  the mask in one call; exec, the seam test tool and the SDK docs use it.
- In the contended-miss loop a probe failure surfaces with 'probing the
  local cache for <addr>' rather than under the lock-acquire context.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/entrypoint-driver branch from 69b7ed2 to c6d614e Compare October 29, 2026 01:18
@raphaelvigee raphaelvigee changed the title feat(exec): exec.entrypoint driver, args passing and --shell feat: entrypoints — invoke a target in your world through the driver's ordinary run Oct 29, 2026
raphaelvigee and others added 5 commits October 29, 2026 03:00
…request state

The registry holds a Weak to the *root* RequestState, which a command drops
when it returns; a remote upload rides a child state sharing the same data,
so at the exit-time drain cancel_all_requests upgraded nothing and a Ctrl-C
or TERM after an entrypoint's tool exited (R27) could not end the stalled
upload: heph waited out the upload's deadline. Cancellation now goes
through a per-request token map, removed when the data (not the root)
drops. Fixes bin-e2e signal_after_the_tool_exits_cancels_the_uploads.

Test: cancel_all_requests_reaches_work_outliving_the_root_state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RLIMIT_CORE=0 does not stop a dump when core_pattern pipes to a collector
(apport, systemd-coredump, as on CI's Ubuntu runners): the kernel skips
the limit check for pipes. reraise now also marks the process
non-dumpable (PR_SET_DUMPABLE 0) on Linux, which no pattern overrides.
Not from this round's changes: the limit alone never covered Linux.

Test: reraise_dies_of_the_signal_without_a_core (re-exec with the core limit
at its maximum; red on a piped core_pattern without the prctl).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
crates/plugin-entrypoint-test-cdylib: a managed driver whose parse declares
an entrypoint and whose run spawns its tool through hplugin_sdk::foreground.
The build job builds it with the shipped cdylibs and uploads it as
test-entrypoint-plugin_<os>_<arch>; bin_e2e downloads test-* alongside
heph*, and the e2e script stages it (both modes) as test-entrypoint-plugin.
Pre-release and perf download heph* only, so it is never published;
tests/ci_gate.rs test_fixtures_are_never_published guards the names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shipped heph loads the test cdylib through its manifest; under a PTY
the tool sees the caller's cwd, env and HEPH_ROOT, args byte for byte and a
tty, its exit code is heph's, and a SIGTERM to heph's pid reaches its trap
exactly once. The in-process seam test stays, its docs point here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…k (R8)

Offline: the pin names a version already in $HOME/.heph/versions (a copy of
the binary under test), so ensure_binary downloads nothing and exec_into
runs. The entrypoint sees HEPH_SELF_UPDATED=1, the lowered soft limit and
the blocked SIGUSR1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@raphaelvigee raphaelvigee added the ci/perf-test Opt in to running the perfbench (Tier A/B) job on this PR label Oct 29, 2026
Replaces R1, at the user's call: an entrypoint's inputs are no longer all
copied. Transitive tools and `read_only_deps` are linked from the shared
stage exactly as a build's are, and held for the run by the same stage
holds; ordinary deps stay copies. `read_only_deps` is accepted on an
entrypoint again.

Accepted: a tool that writes through a linked input changes the shared
stage entry for every build that links it. A `flock` cannot prevent that
(advisory only); the docs tell the author not to write next to a linked
input.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
raphaelvigee and others added 4 commits October 29, 2026 14:47
…ocation

Under 16 concurrent invocations, 4 of 480 failed with `claim invocation
sandbox …: No such file or directory`, exit 125 (found by R24's
measurement). A sibling's sweep can lock the fresh `.owner-pending` before
its creator does and remove the whole directory; the creator then sees
ENOENT from the open or the rename, not only `WouldBlock`. Both now mean
"take another name".

`create_survives_a_concurrent_sweep` failed 3 of 3 before the fix
("claim invocation sandbox …"), passes 5 of 5 after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…king

An entrypoint (R29) or build with transitive tools places every tool file
from the shared stage into its sandbox with `link_tree`. On APFS a hardlink
costs ~0.24 ms (the inode becomes a sibling-linked one and its link count
moves), and every sandbox links the *same* staged inodes, so the cost grows
under concurrency: 100 links took 21 ms alone and ~460 ms each with 16
processes linking at once, and unlinking them at teardown was ~5x a plain
file. `clonefile(2)` gives a new inode sharing the data copy-on-write:
100 clones in ~8 ms, ~64 ms each at 16-way.

The sandbox sees what it saw: a regular file with the staged bytes, mode and
mtime at the same path. What differs on macOS is inode identity (`-ef` no
longer matches the stage file) and that a target which chmods its input
writable and edits it now edits its own copy rather than the stage entry.
`ENOTSUP`/`EXDEV` fall back to the hardlink. Linux keeps `link(2)`, which is
cheap there.

Setup 2 (entrypoint, 100-file transitive tool, warm, darwin/arm64):
58.8 -> 44.9 ms median.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… open

Each output of a multi-output tool is its own input and its own stage entry,
so a 100-file tool is 100 `link_tree` calls per sandbox. Besides the link,
each one walked the read-only entry (an opendir/getdirentries/close per
directory, ~4.5 ms per 100) and opened and closed the origin's list file
(~2 ms per 100).

- `<group>/<hash>.files` caches the entry's node list, written by the first
  per-file consumer (temp + rename, under the shared hold) and read by the
  rest. Its header carries the entry dir's dev/ino/ctime, read under the hold,
  so a list from an earlier materialization is walked over and replaced; a
  list that does not parse is too. Compatible both ways: older binaries never
  read it, and entries without one are walked and get one. `clear_stage`
  removes it with the entry.
- `ListFile` keeps the list open across consecutive inputs that share it
  (one slot, so at most one fd however many inputs a run has). Writes stay
  whole and unbuffered, so lines land in call order and interleave with the
  copy path's own appends as before.

Setup 2 (entrypoint, 100-file transitive tool, warm, darwin/arm64):
44.9 -> 40.6 ms median after the clone change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ks on Linux

The user accepted the divergence (a35c502): APFS hardlinks measured ~3x
slower than clonefile, ~8x under 16 concurrent invocations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
raphaelvigee and others added 2 commits October 31, 2026 15:41
is_entrypoint parses the def, and anything but Ok(true) fell through. Without
args that is fine, since the build path reports the error, but with args the
refusal fired first: a def that failed to resolve read as "is not one".

Also moves the "invoking" log to debug, so it no longer prints at the default
level ahead of the tool's own output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/perf-test Opt in to running the perfbench (Tier A/B) job on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant