Split subprocess launching into Linux and macOS implementations

[cdleary]

This PR builds on @ecpeterson's macOS subprocess support in #4951. I wanted to stack a launcher interface on top, with Bazel selecting the implementation, so the shared subprocess code needs fewer #ifdefs.

Why keep the wrapper?

XLS can be multithreaded. After fork, the child inherits runtime state but only the calling thread survives; POSIX restricts it to async-signal-safe operations until exec succeeds. Using posix_spawn lets libc handle that launch machinery without XLS running its own setup code in that interval. (POSIX fork specification)

Older Linux libc cannot express a child working-directory change through a spawn action. The wrapper provides that capability: posix_spawn starts the helper, which changes directory and calls execvp on the requested command. The helper runs in a fresh executable image, so it can use ordinary C++ diagnostics during setup. The target replaces the helper and keeps its PID. Embedding the helper and executing it from a memfd avoids a runtime dependency on a separately installed helper or Bazel output files.

With a suitable libc and deployment target, the build can select the direct launcher, which records the chdir action and spawns the target immediately.

Implementation selection

subprocess_posix.cc contains the direct-spawn algorithm with no platform conditionals. Bazel selects a small chdir adapter through --//xls/common:subprocess_chdir:

Setting	Implementation
auto (default)	Darwin availability adapter on macOS; subprocess_with_wrapper.cc on Linux for older-libc compatibility.
posix	Standard POSIX.1-2024 chdir action, requiring a compatible toolchain and deployment target.
glibc	glibc's chdir extension, available from 2.29, on Linux.

Only the Darwin adapter contains SDK/runtime availability checks. The standard and glibc adapters each make one API call. The wrapper owns its embedded memfd helper with FileDescriptor; direct builds omit the helper dependency. Both launchers sit behind subprocess_for_os.h, with pipe capture, environment construction, timeouts, and waiting in the shared caller.

Darwin's posix_spawnp can return ENOENT after launching a child when a chdir action is combined with a relative PATH. The direct implementation searches the child's PATH using posix_spawn for each candidate. It handles empty and relative PATH entries, permission errors, and the shell fallback for executable text without a shebang. Five launcher tests exercise these cases.

Stack and related work

1. Support Darwin subprocess working directories with posix_spawn #4951 — Add Darwin subprocess working-directory support.
2. This PR — Separate direct POSIX spawning from the wrapper and cover PATH-search behavior.

This branch contains #4951's commit (d5cc747d4) plus the follow-up commits, and targets main.

Related: #4864 implements non-Linux support by materializing the helper in a temporary file. This approach uses native Darwin spawning without a temporary helper.

Testing

With Bazel 8.7.0:

bazel test --config=ci -c opt --test_output=errors \
  //xls/common:subprocess_test //xls/common:subprocess_for_os_test

To test direct spawning with the hermetic glibc 2.39 toolchain:

bazel test --config=ci -c opt --test_output=errors \
  --platforms=@llvm//platforms:linux_x86_64_gnu.2.39 \
  --//xls/common:subprocess_chdir=glibc \
  //xls/common:subprocess_test //xls/common:subprocess_for_os_test

The resulting Linux binaries require a compatible runtime.

Configuration	Result
macOS 26.6.2 arm64, Darwin adapter, deployment target 11.0	All 19 test cases passed.
macOS 26.6.2 arm64, standard POSIX adapter, deployment target 26.0	All 19 test cases passed.
Linux wrapper, glibc 2.28	All 19 test cases passed (cached).
Linux glibc adapter, glibc 2.39 toolchain, Ubuntu 24.04 container runtime	All 19 test cases passed.

The macOS runs set matching --macos_minimum_os, --host_macos_minimum_os, and --action_env=MACOSX_DEPLOYMENT_TARGET values. The standard adapter run also set --//xls/common:subprocess_chdir=posix. Bazel dependency checks confirmed the wrapper/helper in the Linux default and exactly the glibc adapter with no helper in the Linux direct build.

Older macOS runtimes have not been exercised. Availability guards select the macOS 26 API when available and the older _np API otherwise.

[allight]

Can you instead have the check be for _POSIX_VERSION >= 202409L instead of checking for macos version and name files subprocess_posix and subprocess_with_wrapper? It seems that new versions of glibc & linux have the same addchdir functionality so it would be nice to be able to swap everything over to that once we update our libc.

Otherwise this all looks fine (although merging it might take a while due to internal repo differences.

[cdleary]

@allight Happy to use subprocess_posix and subprocess_with_wrapper and make the direct-spawn implementation reusable on Linux. One wrinkle: macOS still reports _POSIX_VERSION == 200112L, despite supporting the chdir action, and we need runtime availability checks when targeting older macOS versions. Could we retain that small Darwin compatibility adapter while generalizing the implementation selection?

[allight]

SGTM