Repository navigation
feat: entrypoints — invoke a target in your world through the driver's ordinary run - #516
Open
raphaelvigee wants to merge 50 commits into
Open
raphaelvigee wants to merge 50 commits into
raphaelvigee wants to merge 50 commits into
Conversation
…t split D47 of the tool-shims spec: a golden over exec's parse() and apply_transitive() output for a fully populated spec (deps, hash_deps, runtime_deps, tools, env, out, support_files, runner, pass_env, and a transitive closure with tools, deps and hashed/unhashed env), for exec, exec with coreutils, and bash. The exec.entrypoint driver splits parse and apply_transitive into shared helpers; this pins the hashes and input lists first, so that split cannot move an exec def hash unseen. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first slice of the tool-shims spec: an `exec.entrypoint` target runs in
the caller's world — their cwd (or the workspace root), their environment,
their stdio — instead of a sandbox, and `heph run //t:x -- args` runs it.
Driver (crates/plugin-exec, D15, D19, D26, D8): its own EntrypointDef and
def hash; exec's dependency vocabulary (deps, hash_deps, runtime_deps, tools,
env, transitive) through parse/apply_transitive helpers shared with exec by
name, not a branch on the driver name (the D47 golden pins exec's output
across the split). Parse refuses out, cache, codegen, support_files,
scratch, pass_env, runtime_pass_env, runtime_env, credentials (D28), runner
("not supported yet": needs base_env, ABI 0.18.0), workdir other than
"caller"/"root" (D11), unknown fields and wrong types, each naming the field
and target. `args` on exec/bash is refused (D18). The shim label is not
checked at parse (D36).
Engine (D1, D30, D43, D46, I2, I6, I8): the Entrypoint capability is a
builtin-only PluginDriver variant. Engine::invoke resolves the spec, refuses
undeclared args before any dep builds, builds the closure, drains the sqlite
write-behind and pending remote uploads, resolves ${read://} through
resolve_deferred, stages every dep in <home>/stage and holds it, flattens
the tools into a held toolbin stage entry, and hands the driver the private
Invocation token to shape argv/env/cwd. No execute lock. Cache is forced off
from the capability in get_def_inner. An entrypoint is refused as an input
in expand_inputs (deps, runtime_deps, tools, hash_deps, runner, read://,
transitive closures), in multi-target selections, and by an execute
backstop for a direct result_addr.
Holds (I4, D27, proposed in the spec): a shared flock on each stage entry's
<hash>.lock, taken then re-checked (.ready, entry dir; FRWLock checks the
inode), re-materialized on a miss, deduped by path. clear_stage now deletes
.ready before the entry.
Binary: the binary, not the engine, spawns the prepared command, stdio
inherited, env built as OsString (no filter_long_env, D29), and exits with
the child's code or re-raises its signal with RLIMIT_CORE at 0. Process
model is a temporary stand-in for D2: the child stays in heph's process
group, system(3)-style (SIGINT/SIGQUIT ignored while waiting, reset with
SIGTTOU/SIGTTIN in the child).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
D13 of the tool-shims spec: `heph run --shell //t:tool -- a 'b c'` opens an
interactive bash in the tool's world — its env, PATH order (toolbin first)
and cwd — with the closure built and held while the shell lives; heph exits
with the shell's status.
It reuses exec's machinery: the same init.sh template in its own
`{% if entrypoint %}` branch (exec's render is pinned byte for byte by
exec_shell_init_is_byte_identical), and exec's host-PATH lookup for bash
(resolve_on_host_path, now shared with resolve_shell). The command line is
the entrypoint's resolved argv, each word quoted, plus the args; the caller's
args are bash's positional parameters (`bash --noprofile --rcfile <init> -i
-s -- <args>`). `run`/`xrun`/`run1`/`xrun1` take their own args, which
replace the originals for that call; a bare call always means the originals.
`show`/`showl` print the args %q-quoted. No $HEPH_CMD or $HEPH_ARGS.
Deviation from the spec, forced by bash: an interactive bash binds its
positional parameters only after reading its rcfile, so init.sh copies them
at top level (the sourced case) and again in a one-shot PROMPT_COMMAND
before the first prompt, which then restores the caller's.
init.sh is a per-invocation file under <home>/tmp, never the cwd, removed
with the invocation. Undeclared args are refused before anything builds, as
for a plain run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The `exec.entrypoint` driver and its `EntrypointDef` are gone. An
entrypoint is now `entrypoint = True` on an `exec` or `bash` target, with
`args` and `workdir` as exec spec fields and an
`Option<EntrypointMode { args, workdir }>` on exec's def.
- Exec's def hash and serialized def do not move without the flag: the
mode is skipped by both Hash and serde when absent. The D47 golden is
unmodified and green. With the flag the hash covers entrypoint, run,
env, workdir and args; the coreutils version is not folded in (D24).
- With the flag, parse keeps the old refusals (out, cache, codegen,
support_files, scratch, read_only_deps, pass_env, runtime_pass_env,
runtime_env, credentials, runner, workdir other than caller/root), each
naming the field and the target, and skips the workspace runner
default. Without it, `args`/`workdir` are refused, pointing at
`entrypoint = True`. An entrypoint's `env` takes `${read://}`.
- `bash(entrypoint = True)` runs its script through exec's bash
invocation with the caller's args as positional parameters ("$@"),
directly and under `--shell`.
- Engine: `TargetDef.entrypoint` (serde-skipped when false) replaces the
driver capability for cache-off, the input refusal, the execute
backstop, the selection refusal and `invoke`. The `EntrypointDriver`
hook stays, no longer a `Driver` subtrait, registered with
`PluginParts::with_managed_entrypoint_driver`; `invoke` refuses a
flagged def whose driver has no hook (`EntrypointWithoutHook`).
- No ABI change: `target_def_from_pb` sets `entrypoint = false`, so a
cdylib driver cannot declare an entrypoint.
Every `TargetDef` literal gains `entrypoint: false` (compiler-listed).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An entrypoint has one command line, so its `--shell` offers `run`, `xrun`, `show` and `showl` only; `run1`/`xrun1` and their help line are gone. `run`/`xrun` keep the arg override. Exec's `init.sh` render is unchanged (`exec_shell_init_is_byte_identical` passes unmodified), and `entrypoint_shell_init_args_exact` drops its `run1` rows for one asserting `run1`/`xrun1` are not defined. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a hand-written Default for TargetDef (unit raw_def, caching off) and use struct-update syntax in test literals, dropping fields that equal the default. Production literals keep an explicit entrypoint: false. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Production TargetDef literals (driver parse implementations and the wire conversion) now use struct-update syntax too, dropping every field equal to the Default impl. The only explicit entrypoint: false left is the Default impl. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Finish the ..Default::default() pass: engine TargetDef literals still carrying empty inputs/outputs or cache off, the production exec literal, and exec's own def struct, which now derives Default so its test literals state only what they set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rebase onto master brought #519, which made exec's `env` and `runtime_env` `Deferred<String>` maps; the entrypoint hook still read them as `String`s and plugin-exec no longer compiled. Keep the hook's behaviour (raw text for the closure's `runtime_env`, resolved values for `env`) with the new types. The hook itself goes away in the entrypoint rework. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e driver contract
The driver-contract half of the entrypoint rework: an entrypoint runs through
the driver's ordinary `run`, so the contract carries what that run needs.
- `TargetDef.entrypoint: Option<Entrypoint { accepts_args, workdir }>`
replaces the bool; `Workdir` is `Caller`, `Root` or `Sandbox`. Skipped in
`inspect def --json` when unset. Readers only switch to `is_some()`.
- `RunRequest.options: RunOptions { cwd, base_env, args, stdio }`, default for
every build. `RunStdio::Inherit(Foreground { signals, dispositions })` carries
the relay receiver and the dispositions the child execs with. `RunOptions`
is not `Clone`, and its `Debug` prints `base_env` keys only.
- `RunRequest` is `#[non_exhaustive]`, built with `RunRequest::new` plus field
assignment; `with_target` moves a request onto another def (the shell
fallback), naming every field so none can be dropped there.
- `ProcessFailed.exit: Option<Exit>`, `Display` unchanged; exec fills it.
`RemoteProcessFailed` and `process_exit()` read the status back whichever
side of the plugin seam produced it.
- `Exit`, `Signal` and `Disposition` live in `hcore::process`, shared with
`execrunner`, whose `ForegroundExit` is now an alias of `Exit`.
Behaviour is unchanged: nothing sends non-default options yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lugin ABI ABI 0.17.1, an additive patch; MIN_PLUGIN_ABI stays 0.17.0. - `TargetDef.entrypoint` (11): absent decodes to `None`, so a def from an older plugin is never an entrypoint; an unknown workdir is refused. - `ManagedRunRequest.options` (14): `RunOptions` with cwd, env and args as bytes (non-UTF-8 survives), wrapper messages keeping unset apart from empty, and the stdio mode with its dispositions. `run_options_to_pb` refuses non-default options for a def that declared no entrypoint (N9), so the host never sends them to a plugin that would run them as a build. - `RunInFrame.signal` (4): an OS-local signo, never persisted. - `RunOutFrame.exit` (5), outside the oneof: `error` stays the one terminal failure frame, so an older host still prints the message. - `CreateConfig.host_capabilities` (5): the `entrypoints` bit. Absent is no. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…by seam Host (`StableRemoteManagedDriver::dispatch_run`): - encodes `RunOptions` on the start frame, refusing them for a def that declared no entrypoint (N9, at the seam); - the run's request stream is now live: a channel the host owns instead of the fixed `HostItemStream` Vec. The relay forwards each signal from `Foreground::signals` as `RunInFrame.signal`, and the sender is dropped only once the terminal frame has been read (N11); - an `error` frame with an exit status becomes `RemoteProcessFailed`, whose message is the frame's, unchanged. Guest (SDK): - rebuilds `RunOptions`; an inherited run gets no output sinks, and a dedicated thread feeds the rest of the request stream's signal frames into the driver's receiver. The terminal frame never waits on it, and an early end (an older host) is "no signals", not a cancel; - a failure's `ProcessFailed.exit` rides out on `RunOutFrame.exit`; - `create_config_from_bytes` records the host's capabilities per plugin, and every def a guest `parse`/`apply_transitive` returns is refused if it declares an entrypoint the host does not advertise (`require_entrypoint_support` lets a driver check earlier). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…:Inherit `spawn_foreground` runs a program in the caller's terminal: inherited fds 0-2, no setsid, heph's process group (the interim model; D2 comments at each place the shared group matters). In the child, allocation-free, INT/QUIT/TTOU/TTIN go back to SIG_DFL, the host's recorded dispositions apply, and the recorded RLIMIT_NOFILE soft limit is restored. `ForegroundChild::wait` selects over the run's signal channel and the child's exit in one task: the exit is observed without reaping (waitid WNOWAIT, Linux and macOS alike), then relaying stops, the pid is untracked from the supervisor, and only then is the child reaped, so a relayed signal never reaches a reaped pid. The skip decision for terminal-generated signals (R2) is the pure `should_relay`, applied at delivery, where the tool's pid is known. Re-exported from hplugin_sdk::foreground for third-party drivers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l table
Parse (entrypoint_mode.rs): `entrypoint = True` takes `args` and
`workdir = "caller"|"root"|"sandbox"`, and refuses out, cache, codegen,
support_files, read_only_deps (R1), pass_env, runtime_pass_env,
runtime_env, runner ("not supported yet") and scratch outside sandbox
mode, each message naming the field, the target and what to do instead.
Credentials are accepted (E18). A non-bool flag is refused. An exec
entrypoint argv naming `$SRC_x`/`${TOOL_x}`/`$OUT` is refused pointing at
`${src://}` and `driver = "bash"` (R19). The def hash folds in
Some(EntrypointMode) and nothing when unset (golden unchanged); the
engine attribute carries Sandbox. An entrypoint's transitive tools are
copied, not linked from the shared stage (R1).
Run: `run_inner` honours RunOptions generically: cwd override (a missing
directory is named), base_env as the raw bottom env layer with exec's
layers on top and no search_path PATH (bin/ first, coreutils last), args
appended to argv or as bash's "$@" (with a $0 for `-c`, none for a
spilled script), E2BIG named. `RunStdio::Inherit` spawns through
execrunner's foreground spawn and relays the run's signals; a failure
carries `ProcessFailed.exit` and an empty log, never a sandbox handle
(R22). `--shell` renders init.sh's args mode whenever
`RunOptions.args.is_some()` (E11), and forwards only TERM/HUP to the
session's group (Open 7). Without args init.sh is byte-identical.
The prepare hook (entrypoint.rs) stays until the engine and binary stop
calling it (layer 3). The e2e caller-cwd test moves its `sh -c` script
to `driver = "bash"`, as R19 now requires.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sandbox_guards Every stage entry a run links to is held from link until the run's guards drop (N3, E14). `stage_and_link` materializes, takes a shared flock on the entry's `<hash>.lock`, re-checks `.ready` under it (bounded retries when a gc lands in the window), then links, and returns the hold. The managed OS runner hands the holds out through `RunResponse.sandbox_guards`, which the engine drops after the cache write; on any failure they drop with the frame. `clear_stage` already tries the exclusive lock, skips a held entry, and removes `.ready` before the tree. The invocation-only `StageHolds`/toolbin machinery of the prepare hook goes: an invocation copies its closure (R1) and stages nothing of its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`execute`'s sandbox block becomes `Engine::run_in_sandbox`, shared with `invoke` (R12, E7) and parameterised by a `SandboxLease`: a build's per-address directory (generation-checked stale removal; kept for diagnostics on failure) or an invocation's own one (created empty and exclusively; removed on every exit). It returns heph's failure and the driver's run apart, and refuses non-default run options for a def that is not an entrypoint (N9). Invocation sandboxes (R11, N10, E20) live at `<home>/sandbox/<pkg>/__invoke/<name>/<uuid>`: an exclusive leaf create retried on AlreadyExists only, an `.owner` file locked before it gets its name, the lock held until the tree is gone. A crash leftover — `.owner` present and free — is swept by the next invocation of the target and by `heph gc` (`GcStats.invoke_sandboxes_removed`), never through a symlink and never into a build's `__target_*` sandbox. No generation claim is taken, so the never-pruned claim map does not grow per invocation (Open 9). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t its readers A request that missed on a target while another built it used to block on the revision's write lock, which waits for every reader to drain. Once the builder downgraded, its consumers' riding reads kept the lock — for an entrypoint, for as long as the tool runs, since its closure carries them — so the loser waited for the winner's tool to exit instead of sharing the build (N10). The miss path now tries the write lock, and when it is taken waits only for the target's gateway (free once any build of it ends), re-probes under a read and shares a hit; a miss still contended was a passing prober, so it backs off (5–100ms, cancellable) and tries again. `concurrent_invocations_run_in_parallel` timed out after 120s before this change, with one tool running and the other fifteen requests parked in `ResultLock::write`; it passes in under a second after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… run `Engine::invoke` reads `accepts_args` from `TargetDef.entrypoint`, refuses undeclared args before any dep builds, builds the closure, drains the cache writes it queued, and runs the shared sandbox block with an invocation lease and `RunOptions`: cwd from the workdir (caller's, the root, or none for the sandbox), the caller's env with `HEPH_ROOT` appended (R17), args only when declared, and the caller's terminal with the relay receiver and dispositions the caller hands in (R9: the engine installs no handler). It takes no execute lock, no worker permit and no memoizer slot, writes no cache, and removes its sandbox before returning, on every exit. One `info` event names the invocation (R20). It returns an `InvokeOutcome` (R4): `Exited(Exit)` when the tool ran — read from the driver run's own `ProcessFailed.exit`, never a dep's — or `Failed(err)` for a heph-side failure after resolution; `Err` stays a failure before the target resolved. The entrypoint attribute is the parse's (R13): restored when a driver's `apply_transitive` drops it, refused when it names a different one. The FUSE bridge always runs an entrypoint on the OS runner (Open 8). The e2e suite moves to the run path and gains the spec's rows: the refusal table, workdir modes with scratch, 16 concurrent invocations over two engines with gc running, the sandbox removed on every exit, copies not links, credentials, and the transitive tamper rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR #516's parallel universe goes, now that an entrypoint runs through its driver's ordinary `run`: `hplugin::driver::entrypoint` (`EntrypointDriver`, `Invocation`, `PrepareRequest`, `PreparedCommand`, `StagedInput`, `Caller`), `PluginDriver::ManagedWithEntrypoint` and its registration, exec's prepare hook, and execrunner's `run_foreground`/`ForegroundExit`. exec and bash register as plain managed drivers again. `heph run` calls `Engine::invoke` inside the app, the TUI paused for input for the run, and exits as the tool did. The signal relay, the recorded dispositions, the suppression guard and exit 125 are still to come (`TODO(entrypoint-L4)`). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ches exec on macOS A child forked while another thread of a fresh process was initializing libSystem's libnotify globals died in the atfork child handler (_os_once_gate_corruption_abort, SIGKILL) before exec. std then reads EOF on its exec-error pipe and reports a successful spawn: a missing program "spawned", a real one is "killed by signal 9". This was the execrunner foreground_spawn_missing_program_is_not_found flake (~1 in 3 runs of the lib tests, racing foreground_spawn_closed_channel_is_not_a_cancel). hproc::fork_safety::prime() initializes the globals once before any fork that goes through pre_exec (proc_exec on macOS, the foreground spawn). Regression test first_concurrent_foreground_spawns_reach_exec races two first spawns in 30 fresh processes: red on attempt 1 without the prime. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pgrade hcore::process::record_inherited() records the dispositions of every signal heph touches (INT, QUIT, HUP, TERM, PIPE, TTOU, TTIN, USR1, USR2, WINCH) and the open-file soft limit, before main changes any of them (R8). PIPE is recorded as default (X6: Rust ignores it before main). The foreground spawn reads the NOFILE record from there instead of its own static. The self-upgrade's execv puts the recorded state back first (restore_inherited_for_exec, in a pre_exec), so the pinned binary records the caller's dispositions and limit rather than the TTOU/TTIN ignore and the raised limit the first binary applied. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replaces the single suppression bool for entrypoints (R9). While any hcore::shutdown::TransparencyGuard is alive the shutdown trigger drops every press, its consumer drops presses queued before the hold, and the SIGQUIT dump, the per-thread SIGUSR1 backtrace and the SIGUSR2 profile dump stand down. Refcounted, so two invocations never end each other's hold. Foreground gains a host-only Handover hook, fired once right before the driver that spawns the program is called: by the managed layer after the inputs are materialized (host side, so it covers cdylib drivers), or by the engine for a plain driver (engine Driver.managed). Deps building and the closure materializing therefore stay cancellable with Ctrl-C. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… it did - The relay's listener (bootstrap::spawn_signal_relay) forwards TERM, HUP, INT, QUIT, USR1, USR2 and WINCH into InvokeRequest::with_signals, unfiltered: the driver applies the terminal skip (R2). heph no longer dies of them while the tool runs. - The recorded dispositions go to with_dispositions; the handover takes the transparency hold until exit (R9). The TUI pauses with PauseFor::Output so a Ctrl-C while the closure materializes still cancels; the hold owns it from the handover on. - Exit mapping (R4): the tool's code as is, its signal re-raised, 125 with the box for a heph-side failure after resolution, 130 for a Ctrl-C before the tool started, 1 before resolution (run::InvokeFailed). - R18: an entrypoint reached by a selection names the selection's size once the walk finished (keep-going); `heph run -- a` and `heph run //x word` are refused with the `--` spelling to use. - R20: the TUI header says "Invoking"; inspect def adds "runs_as": "exec, entrypoint (uncached)" for an entrypoint only. - Docs: docs/ENTRYPOINTS.md and the run --help text (exit codes, signals, --shell + show, $WORKSPACE_ROOT vs $HEPH_ROOT). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sidecar shares heph's process group, so the terminal's Ctrl-C, Ctrl-\, Ctrl-Z and hangup, and any kill -<pgid>, reached it too, and the default action killed it. With an entrypoint holding the terminal the tool keeps running after a Ctrl-C (it is the tool's), and a later kill -9 of heph then had no supervisor left to reap it. It now ignores INT, QUIT, HUP, TERM, TSTP, TTIN, TTOU, USR1 and USR2; it still ends when heph's end of its socket closes. bin-e2e supervisor_survives_ctrl_c_and_reaps_after_kill_9 (entrypoint_pty) times out waiting for the reap without this, passes with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A shell runs on a PTY of its own and ignores RunOptions.stdio: the wrapper's streams are how the terminal reaches that PTY. The shared sandbox block dropped them for every Inherit run, so heph run --shell on an entrypoint showed nothing and read nothing. They are now passed on to a shell run, and the invocation's wrapper hands them in (TtyReader, as a build's --shell does). The wrapper is also installed without a TUI, so the stall notice is suppressed while any tool runs. The heph-side failure path wraps finalize! in its own block: its early ? returned the bare error, which then exited 1 instead of 125. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
entrypoint.rs: entrypoint_keeps_inherited_dispositions (plain caller vs
trap '' INT TTOU TTIN), entrypoint_signal_to_heph_is_forwarded (TERM and USR1
once each, a TERM that kills the tool leaves none), error_frame_without_exit_
exits_125_with_box (missing program, failing dep, refused arg -> 125; typo ->
1), sigkilled_heph_tool_reaped_and_sandbox_swept, ctrl_c_during_dep_build_
cancels (130, tool never ran).
entrypoint_pty.rs (vt100 cells via common::pty): entrypoint_tui_pauses_for_tool
("Invoking" label, tty on fds 0-2, cursor restored), entrypoint_ctrl_c_goes_
to_tool (two presses each delivered once, USR1 relayed once, heph silent,
exit 7 not 130), supervisor_survives_ctrl_c_and_reaps_after_kill_9.
shell_pty.rs entrypoint_shell_pty: the init file lives in the invocation
sandbox now; adds the show row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cdylib_entrypoint_runs: a third-party managed driver exported with named_managed_driver and decoded by the host loader's decode_drivers, with the host capability bit recorded from encoded CreateConfig bytes as a cdylib's create entry does. Its parse declares the entrypoint, its run spawns through hplugin_sdk::foreground honouring RunOptions. Engine::invoke drives it: caller cwd, caller env + HEPH_ROOT, args byte for byte, exit code as the outcome, and a SIGTERM pushed into the request's signal channel reaching the tool's trap over the RunInFrame.signal relay. entrypoint_survives_apply_transitive now runs every row in-process and behind the seam (R13's cdylib leg). In-process rather than bin-e2e with a test cdylib, following plugin_fn_declarations.rs: what an entrypoint adds rides the run's protobuf frames, so dlopen has nothing entrypoint-specific to reject. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… invoke start Before the tool is about to be spawned heph is an ordinary run again: TERM and HUP end it, Ctrl-C cancels, SIGQUIT dumps diagnostics and SIGUSR2 a profile. The relay's channel still exists from the start so the request carries its receiver, but its listeners are installed by the handover, so nothing a user sent to a slow dep build is queued for a tool that does not exist yet (a QUIT to inspect it no longer kills the tool later). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on Foreground The recorded RLIMIT_NOFILE soft limit was restored only from a static in the host's execrunner, so a plugin driver's foreground spawn (whose copy of that static is never set) kept heph's raised limit (D5). It now rides the run as Foreground.nofile and an additive RunOptions.nofile on the wire (still ABI 0.17.1, unreleased), and ForegroundCommand::with_nofile sets it. The caller's signal mask is recorded with the dispositions and restored the same way (Foreground.blocked, RunOptions.blocked_signals): std clears the mask in every child before its pre_exec, so without it a tool started under a blocked signal ran with none blocked. The self-upgrade's exec restores it too. A cancel seen after the runner's prepare now spawns nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runner agent, telemetry and fsutil fork too, and never called fork_safety::prime(); priming once while the process is single-threaded covers every fork the binary makes. The two spawn sites keep their own call for processes that never run heph's main (tests, embedders). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
R1 was enforced only by exec's apply_transitive dropping READ_ONLY; the managed OS runner linked any read-only input from the stage regardless of the def, so another driver's entrypoint could write through a link into the shared stage entry. run_inner now treats the stage as absent for a def with `entrypoint` set: every input is a writable copy and nothing is held. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…giene - Drain only the local cache's queued writes before the tool starts, and cancellably (biased to the cancel): remote uploads run on beside the tool and are drained at exit like any command's, instead of delaying the spawn by up to their timeout; a Ctrl-C while a write is stuck ends the run before the handover. - Refuse an entrypoint named `__target_*`: its invocations would land in the build sandbox of `//<pkg>/__invoke:<x>`. - Remove the lock file last, so a teardown that fails partway stays sweepable; sweep an unlocked `.owner-pending`-only dir (a crash between the lock and the rename), the creator retrying another name if a sweeper locks its fresh one first; never walk into a live invocation's tree. - A driver-reported exit status no failed process can have (code 0 or outside 1-255, a stop/ignore-class signal) is heph's failure (125, with the driver's message), not a status heph tries to exit with. Tests: cancel_before_the_handover_spawns_nothing, abandoned_invocation_is_reclaimed, entrypoint_name_with_build_sandbox_prefix_is_refused, failure_status_table, and new rows in stale_invoke_sandbox_swept / sweep_all_finds_every_package. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
std ignores SIGPIPE before main, so the recorded disposition was always Default (accepted limit X6). A pre-main constructor (.init_array / __mod_init_func) now notes it before lang_start; the tool and the self-upgraded binary get it back. X6 is no longer a limit. bin-e2e: the PIPE rows in entrypoint_keeps_inherited_dispositions, entrypoint_keeps_callers_open_file_limit_and_mask, term_or_hup_during_dep_build_ends_heph, quit_during_dep_build_does_not_reach_the_tool, entrypoint_kill_under_a_terminal (TERM once, INT dropped under X5, group TERM at least once); every non-PTY wait is bounded and kills heph on timeout (common::wait_output, Workspace::run). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A Ctrl-C accepted before the handover cancels even when its consumer reads it after: presses are gated at trigger time only. - diag's per-thread dump installs its SIGUSR1 handler once and never over someone else's (the relay's listener). - ForegroundChild's drop kills, untracks, then reaps, as wait does. - RunOptions::is_default destructures, so a new field cannot be missed (N9). - RemoteProcessFailed is #[non_exhaustive] with a constructor. - Driver contract docs: an entrypoint's run may be one of several in flight for one addr, each in its own sandbox_dir (N10); when the handover fires relative to run. - The shell relay's killpg comment states its residual pgid-reuse window. - first_concurrent_foreground_spawns_reach_exec loops only on macOS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he loop The user's decision on D1: an entrypoint's dep result locks stay held for the whole run, so the contended-miss loop (try_write; else wait on the gateway, re-probe under a read, share a hit; else back off) stays. - The whole contended wait is one acquire_with_notice scope: a target held by another process or request gets one ResultLockWaitStart (with the holder where known) after RESULT_LOCK_NOTICE, whichever arm it waits in — the backoff included, which emitted nothing — and one ResultLockWaitEnd. - After a miss under the gateway, try the write lock before backing off; the backoff is jittered so racing waiters do not retry in lockstep. - The first try_write carries 'acquiring result lock for <addr>'. - The loop is a generic contend_for_write so each arm is unit-tested: winner failed -> wins try_write, late hit shared, backoff until free, cancel inside the backoff sleep, and the notice in the gateway, probe and backoff arms (silent under the threshold). e2e: dep_held_by_another_engine_announces_the_wait. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…X5, NOFILE
- The R19 refusal message and ENTRYPOINTS.md no longer claim exec 'has no
shell' as the reason: the variable is refused even inside sh -c because
heph cannot tell an intended expansion from literal text; bash (quoting a
literal '$OUT') or ${src://...} is the way.
- Before the tool starts heph is an ordinary run (TERM/HUP end it, QUIT
dumps), nothing is queued for the tool; materializing is silent while
the TUI is paused but still cancellable.
- X5 (kill -INT/-QUIT/-HUP of heph's pid while it owns the terminal is
dropped) and the group-signal double delivery are documented; X6 is gone
(an inherited SIGPIPE ignore is kept), as is the builtin-only NOFILE note.
- FUSE is ignored for entrypoints; runs_as is for people, programs read
target_def.entrypoint (now pinned by the inspect def test).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…R27) Transparency was held, and the relay kept swallowing TERM/HUP, until heph exited, so a post-tool drain of a stalled remote upload (up to UPLOAD_DEADLINE) ignored everything but SIGKILL. Now the handover's TransparencyGuard is handed to hcore::shutdown::tool_exited once invoke returns: when no other tool runs, Ctrl-C, TERM and HUP cancel what heph is still finishing (the relay's listeners call back into cancel_all_requests instead of forwarding), silently — no message, no hard abort, no 130 — and heph exits with the tool's status. The refcount still keeps a second invocation's tool transparent. The relay arms all listeners before spawning their tasks; the instant inside tokio's signal() is documented. Tests: core tool_exited_ends_only_its_own_transparency; bin-e2e signal_after_the_tool_exits_cancels_the_uploads (an S3 endpoint that accepts and never answers; INT and TERM each end heph within 20s with exit 3). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…error context - hcore::process::CallerState, implemented by Foreground, lets ForegroundCommand::with_foreground(&fg) apply dispositions, nofile and the mask in one call; exec, the seam test tool and the SDK docs use it. - In the contended-miss loop a probe failure surfaces with 'probing the local cache for <addr>' rather than under the lock-acquire context. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
raphaelvigee
force-pushed
the
raphaelvigee/entrypoint-driver
branch
from
October 29, 2026 01:18
69b7ed2 to
c6d614e
Compare
…request state The registry holds a Weak to the *root* RequestState, which a command drops when it returns; a remote upload rides a child state sharing the same data, so at the exit-time drain cancel_all_requests upgraded nothing and a Ctrl-C or TERM after an entrypoint's tool exited (R27) could not end the stalled upload: heph waited out the upload's deadline. Cancellation now goes through a per-request token map, removed when the data (not the root) drops. Fixes bin-e2e signal_after_the_tool_exits_cancels_the_uploads. Test: cancel_all_requests_reaches_work_outliving_the_root_state. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RLIMIT_CORE=0 does not stop a dump when core_pattern pipes to a collector (apport, systemd-coredump, as on CI's Ubuntu runners): the kernel skips the limit check for pipes. reraise now also marks the process non-dumpable (PR_SET_DUMPABLE 0) on Linux, which no pattern overrides. Not from this round's changes: the limit alone never covered Linux. Test: reraise_dies_of_the_signal_without_a_core (re-exec with the core limit at its maximum; red on a piped core_pattern without the prctl). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
crates/plugin-entrypoint-test-cdylib: a managed driver whose parse declares an entrypoint and whose run spawns its tool through hplugin_sdk::foreground. The build job builds it with the shipped cdylibs and uploads it as test-entrypoint-plugin_<os>_<arch>; bin_e2e downloads test-* alongside heph*, and the e2e script stages it (both modes) as test-entrypoint-plugin. Pre-release and perf download heph* only, so it is never published; tests/ci_gate.rs test_fixtures_are_never_published guards the names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shipped heph loads the test cdylib through its manifest; under a PTY the tool sees the caller's cwd, env and HEPH_ROOT, args byte for byte and a tty, its exit code is heph's, and a SIGTERM to heph's pid reaches its trap exactly once. The in-process seam test stays, its docs point here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…k (R8) Offline: the pin names a version already in $HOME/.heph/versions (a copy of the binary under test), so ensure_binary downloads nothing and exec_into runs. The entrypoint sees HEPH_SELF_UPDATED=1, the lowered soft limit and the blocked SIGUSR1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 29, 2026
Replaces R1, at the user's call: an entrypoint's inputs are no longer all copied. Transitive tools and `read_only_deps` are linked from the shared stage exactly as a build's are, and held for the run by the same stage holds; ordinary deps stay copies. `read_only_deps` is accepted on an entrypoint again. Accepted: a tool that writes through a linked input changes the shared stage entry for every build that links it. A `flock` cannot prevent that (advisory only); the docs tell the author not to write next to a linked input. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ocation
Under 16 concurrent invocations, 4 of 480 failed with `claim invocation
sandbox …: No such file or directory`, exit 125 (found by R24's
measurement). A sibling's sweep can lock the fresh `.owner-pending` before
its creator does and remove the whole directory; the creator then sees
ENOENT from the open or the rename, not only `WouldBlock`. Both now mean
"take another name".
`create_survives_a_concurrent_sweep` failed 3 of 3 before the fix
("claim invocation sandbox …"), passes 5 of 5 after.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…king An entrypoint (R29) or build with transitive tools places every tool file from the shared stage into its sandbox with `link_tree`. On APFS a hardlink costs ~0.24 ms (the inode becomes a sibling-linked one and its link count moves), and every sandbox links the *same* staged inodes, so the cost grows under concurrency: 100 links took 21 ms alone and ~460 ms each with 16 processes linking at once, and unlinking them at teardown was ~5x a plain file. `clonefile(2)` gives a new inode sharing the data copy-on-write: 100 clones in ~8 ms, ~64 ms each at 16-way. The sandbox sees what it saw: a regular file with the staged bytes, mode and mtime at the same path. What differs on macOS is inode identity (`-ef` no longer matches the stage file) and that a target which chmods its input writable and edits it now edits its own copy rather than the stage entry. `ENOTSUP`/`EXDEV` fall back to the hardlink. Linux keeps `link(2)`, which is cheap there. Setup 2 (entrypoint, 100-file transitive tool, warm, darwin/arm64): 58.8 -> 44.9 ms median. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… open Each output of a multi-output tool is its own input and its own stage entry, so a 100-file tool is 100 `link_tree` calls per sandbox. Besides the link, each one walked the read-only entry (an opendir/getdirentries/close per directory, ~4.5 ms per 100) and opened and closed the origin's list file (~2 ms per 100). - `<group>/<hash>.files` caches the entry's node list, written by the first per-file consumer (temp + rename, under the shared hold) and read by the rest. Its header carries the entry dir's dev/ino/ctime, read under the hold, so a list from an earlier materialization is walked over and replaced; a list that does not parse is too. Compatible both ways: older binaries never read it, and entries without one are walked and get one. `clear_stage` removes it with the entry. - `ListFile` keeps the list open across consecutive inputs that share it (one slot, so at most one fd however many inputs a run has). Writes stay whole and unbuffered, so lines land in call order and interleave with the copy path's own appends as before. Setup 2 (entrypoint, 100-file transitive tool, warm, darwin/arm64): 44.9 -> 40.6 ms median after the clone change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ks on Linux The user accepted the divergence (a35c502): APFS hardlinks measured ~3x slower than clonefile, ~8x under 16 concurrent invocations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
is_entrypoint parses the def, and anything but Ok(true) fell through. Without args that is fine, since the build path reports the error, but with args the refusal fired first: a def that failed to resolve read as "is not one". Also moves the "invoking" log to debug, so it no longer prints at the default level ahead of the tool's own output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the entrypoints spec (draft 3, with board round-1 resolutions R1–R24 and round-2 decisions R25–R27, and R29 replacing R1): https://claude.ai/artifact/LFRqMFJj2yZZrsQT8eGp1b
An entrypoint target is run in your world instead of being built.
heph run //scripts:migrate -- --dry-runbuilds and caches what the script needs, then runs it uncached, in your cwd, with your environment and your terminal, and exits with its status.This PR is a rework of its own first version. The prepare hook (
EntrypointDriver,PreparedInvocation, and the spawn in the binary) is gone. An entrypoint now runs through the driver's ordinaryDriver::run/run_shell, with a few generic additions to the contract.What changed
Contract (
crates/plugin,crates/core/src/process.rs)TargetDef.entrypoint: Option<Entrypoint { accepts_args, workdir }>.RunRequest.options: RunOptions { cwd, base_env, args, stdio }, wherestdioisRunStdio::Inherit(Foreground { signals, dispositions, nofile, blocked }).ProcessFailed.exit: Option<Exit { Code, Signal }>.#[non_exhaustive]and have constructors.ABI 0.17.1 (additive;
MIN_PLUGIN_ABIstays 0.17.0)TargetDef.entrypoint= 11,ManagedRunRequest.options= 14 (nofile= 6 andblocked_signals= 7 inside it),RunInFrame.signal= 4, andRunOutFrame.exit= 5.exitsits outside the oneof, anderrorstays the terminal frame.CreateConfig.host_capabilitiestells a plugin whether the host supports entrypoints; on a host that doesn't, the SDK refuses to declare one.exec/bash
entrypoint = True,args = True, andworkdir = "caller"|"root"|"sandbox".parserefuses incompatible fields (the spec's refusal table).run_innerhonoursRunOptionsgenerically: cwd, caller env as the bottom layer, args appended ("$@"in bash), and an inherited-stdio arm.--shellhas an args mode: positional parameters,run/xrunoverride them, and there is norunN.execrunner foreground spawn
RLIMIT_NOFILE.waitid(WNOWAIT); the relay then stops, the pid is untracked and only then reaped, so no signal reaches a reaped pid. The supervisor registers the pid.Engine
Engine::invokeandrun_in_sandboxgive execute and invoke one shared sandbox block (SandboxLease).<home>/sandbox/<pkg>/__invoke/<name>/<uuid>. It is created exclusively, guarded by a flocked.owner, always removed, and crash leftovers are swept by the next invocation and byheph gc.read_only_depsare linked from the shared stage and held for the run; ordinary deps are copies.read_only_depsis accepted on an entrypoint. Accepted: a tool writing through a linked input changes the shared stage entry.clonefileclones on macOS and hardlinks on Linux; each stage entry caches its file list (<hash>.files).69b7ed2d): bare entrypoint 15.2 vs 14.1 ms; 2,000 siblings 21.8 vs 20.6 ms; ~100-file transitive tool 40.6 vs 21.9 ms; 16 concurrent with that tool 575 vs 89 ms; an SDK-sizedread_only_depslinked in 40 ms. A cold 1,000-package Go build is unchanged against master. The remaining tool-closure gap comes from laying files out per invocation, which R29 requires; it is accepted. The measurement also found a sweep/claim race (~1 in 120 concurrent invocations exited 125), fixed ind31ba6dcwith a stress test that failed 3/3 before the fix.apply_transitive(R13).stage_and_linktakes a shared hold and returns it throughsandbox_guards;clear_stagehonours it (try-lock, remove.readyfirst). This also fixes an existing race between gc and a build.Binary
mainheph records the state it inherited: dispositions (SIGPIPE via a pre-main constructor), signal mask and NOFILE. The self-upgradeexecvpasses that state on to the pinned binary.heph runexit codes:last = true.docs/ENTRYPOINTS.md.Also fixed along the way
hproc::fork_safety::prime()is now called at the top ofmainand at the spawn sites.kill -9of heph left the tool unreaped.contend_for_write, jittered backoff).Release notes
heph runon anentrypoint = Truetarget runs it in your terminal, directory and environment. Exit status: the tool's code, or heph dies of the tool's signal; 125 for heph's own failure after the target resolved (dep, refused argument, closure, credentials, spawn, or a driver reporting an impossible exit status); 130 for Ctrl-C before the tool started; 1 before resolution.heph run -- //xno longer names a target: everything after--is the tool's (last = true). Writeheph run //x -- args.heph run //x wordis refused, with a message naming the--spelling.kill -INT/-QUIT/-HUP <heph pid>sent while heph is the terminal's foreground is dropped, and a signal sent to the whole group may reach the tool twice.execv.entrypoint/args/workdiron exec/bash needs a heph with entrypoints; an older heph fails to parse it. An entrypoint named__target_*is refused.RunRequest,RunOptions,Foreground,EntrypointandRemoteProcessFailedare#[non_exhaustive]with constructors.ProcessFailedgainsexit. UseForegroundCommand::with_foreground(&fg)to apply the caller's state.heph inspect defadds"runs_as": "<driver>, entrypoint (uncached)"for an entrypoint. It is a human string; programs key ontarget_def.entrypoint.Deviations from the spec
cdylib_entrypoint_runshas two layers.dlopen(crates/bin-e2e/tests/cdylib_entrypoint.rs): it loads a new test-only cdylib,crates/plugin-entrypoint-test-cdylib, into the shipped binary through a manifest. Under a PTY it asserts the caller's cwd and env,HEPH_ROOT, a tty, argsa/b c, exit 42 passing through, and SIGTERM to heph's pid reaching the tool exactly once.e2escript and the CI build job build and stage the cdylib astest-entrypoint-plugin_*. It is never published:tests/ci_gate.rstest_fixtures_are_never_publishedguards that.crates/e2e: kept, because it cheaply covers more rows. The cdylib leg ofentrypoint_survives_apply_transitiveruns only there.Handoverhook after materialization, not "before the fork", so Ctrl-C still cancels while the closure materializes. The TUI pauses withPauseFor::Output, notInput.$SRC/$TOOL/$OUT, and$SRC_xeven inside an execsh -celement, because heph can't tell expansion from literal text. The message says to usedriver = "bash"or${src://…}.Workdiris exhaustive, not#[non_exhaustive]: the host matches on it. A cdylib failure surfaces asRemoteProcessFailed { message, exit }.fuse.enabledis ignored for it, and the docs say so.heph runhas no--json, so there is nothing to shape.--shellsession forwards only TERM and HUP, to its pgid.Not done / follow-ups
exec_intorow now exists:self_upgraded_heph_keeps_callers_open_file_limit_and_mask(bin-e2e). It runs offline: a copy of the binary under test is pre-staged as the pinned version in the test's HOME. Not shown red against a binary without the restore.ci/perf-test, so CI's Perf job (baseline vs candidate heph-bench) is the measurement. It covers the general build paths (D1, N3), not R24's entrypoint-specific setups.signal_after_the_tool_exits_cancels_the_uploads(bin-e2e) failed on its first CI run, as did the SIGQUIT row ofentrypoint_signal_death_is_reraisedon Linux. Both are fixed:cancel_all_requestsnow reaches work that outlives the root request state; the upload ran on a child state the registry no longer found.RLIMIT_CORE0 doesn't stop a pipedcore_pattern, so heph also callsprctl(PR_SET_DUMPABLE, 0).entrypointbin-e2e file passes locally on macOS (16/16). Linux CI is the check for the core fix.Board
sh -cwording of the refusal message) is fixed in 1fb2cf1.Tests
lintpasses on HEAD. Spot runs passed for these crates: core, proc, execrunner, engine, driver-support, driver-bridge, plugin, plugin-abi (convert), plugin-sdk, plugin-stabby, plugin-exec, selfupdate, builtins, plugin-oci, and the root crate's lib, bins and supervisor tests.e2e and bin-e2e files that passed:
e2e --test entrypoint(25 tests), plus engine_core, clean, credential, scratch, tool_env, deferred, group, query, oci_image, http_fetch and plugin_fn_declarations.entrypoint,entrypoint_ptyandshell_pty, at 1fb2cf1. The one test added after that hasn't run yet; see "Not done".🤖 Generated with Claude Code