Repository navigation
[mutate] Windows: forward-slash outer argv[0] fails to launch per-mutant children, every mutant silently reports killed #1916
Description
Correction, see the comment below for the full account: the mechanism this report originally described — CliConfiguration.php:64's hardcoded literal reaching the per-mutant child — is wrong. That literal is local to fromArguments(), discarded after building an ArgvInput for option parsing, and never returned. The per-mutant child's command is actually built from the outer --mutate process's own argv[0] (via MutationTestRunner::setOriginalArguments()), so the fix is at the call site (e.g. an @php-prefixed Composer script on Windows, which normalizes the separator), not in this project. The reproduction, symptom, and impact described below are all still accurate; only the attributed cause is not.
Filing here because issues are disabled on pestphp/pest-plugin-mutate. Related but distinct from #1843, which is a different root cause (a Symfony Process deadlock inside a mutant's own test) reaching the same class of symptom — any non-isSuccessful() process outcome gets scored as a kill rather than reported as its own state.
What:
- Bug Fix
Description:
On Windows, CliConfiguration::fromArguments() hardcodes the per-mutant child process's argv[0] as the literal forward-slash string 'vendor/bin/pest':
// src/Support/Configuration/CliConfiguration.php:64
$filteredArguments = ['vendor/bin/pest'];This string is reused verbatim in MutationTest::start() to build every per-mutant Process:
// src/MutationTest.php:85-93
$process = new Process(
command: [
...$originalArguments, // starts with the hardcoded 'vendor/bin/pest'
'--bail',
'--filter="'.implode('|', $filters).'"',
],
env: $envs,
timeout: $this->calculateTimeout(),
);On Windows, a forward-slash relative path in the first element of a Symfony Process array command fails to launch. Isolated in a minimal reproduction against symfony/process 7.4.13, same working directory, only the separator and extension varied:
'vendor/bin/pest' exit 1, stderr: 'vendor' is not recognized as an internal or external command, operable program or batch file.
'vendor\bin\pest' exit 0, runs correctly
'vendor/bin/pest.bat' exit 1, same error
'vendor\bin\pest.bat' exit 0, runs correctly
Only the separator matters; the .bat extension does not change the outcome either way.
Because MutationTest::hasFinished() reads $this->process->isSuccessful() and maps a non-zero exit to MutationTestResult::Tested (killed) rather than treating a launch failure as its own state, every mutant whose covering-test child fails to launch this way is reported as killed, indistinguishable from a genuine kill:
// src/MutationTest.php:139-150
if ($this->process->isSuccessful()) {
$this->updateResult(MutationTestResult::Untested);
// ...
return true;
}
$this->updateResult(MutationTestResult::Tested);Reproduction
On Windows, running --mutate --covered-only --path=<any file> against a real project: every mutant reports as tested/killed, with a score of 100%, and real (non-instant) per-mutant wall-clock time — because each child process takes roughly 1.5-2s to fail to launch and report the error, which reads as plausible execution time. Confirmed by disabling a test known to kill a specific mutant and re-running: the mutant still reports killed, because the process that should have run that test never launched at all.
Instrumented MutationTest.php directly (two fwrite(STDERR, ...) calls printing the constructed command, exit code, and isSuccessful()) and ran a 38-mutant pass on one file: all 38 child processes failed to launch ('vendor' is not recognized...), all 38 were marked killed, reported score 100.00%.
Impact
This is not a narrow edge case — it affects every --mutate invocation on Windows, regardless of how the outer pest --mutate ... command was invoked (bare, absolute path, .bat, through composer exec, through a composer.json script). The outer invocation's form is irrelevant, because the failing command is built from the hardcoded literal in CliConfiguration::fromArguments(), not from the outer process's own argv. On an affected machine, mutation testing produces a plausible-looking, fully fabricated 100% for every run, with no error and no indication anything is wrong.
Suggested fix (superseded, see correction comment)
The CliConfiguration.php change originally proposed here would not fix anything, since that code path is never spawned. What would help: MutationTest::start() normalizes $originalArguments[0]'s path separator (DIRECTORY_SEPARATOR, or Symfony\Component\Filesystem\Path::normalize()) before building the per-mutant Process, so the child's launch does not depend on how the outer process happened to be invoked.
Environment
pestphp/pest-plugin-mutatev3.0.5symfony/process7.4.13- Windows 11, Git Bash / MSYS2, PHP 8.4.20 (Herd)
Activity
joshdaugherty commentedon Sep 15, 2026
Correcting the mechanism claimed in this report. The observed symptom (forward-slash
argv[0]fails to launch on Windows, silently scored as a kill) is real and reproduces. The attribution toCliConfiguration.php:64's'vendor/bin/pest'literal is wrong.That literal is local to
CliConfiguration::fromArguments(). It is used only to build aSymfony\Component\Console\Input\ArgvInputfor parsing the mutate-specific CLI flags (--path,--covered-only, and so on), and is discarded there —fromArguments()'s ownreturn $arguments;returns the original argument list passed in, never$filteredArguments. It never reachesMutationTest::start().What actually reaches
MutationTest::start()is$originalArguments, set viaMutationTestRunner::setOriginalArguments()fromMutate::handleArguments($arguments)'s return value — which is the outer--mutateprocess's own argv (minus the optionsfromArguments()consumed, plus an appended--coverage-php=…). So the per-mutant child'sargv[0]is inherited from however the outer process was actually invoked, not from a hardcoded string.Confirmed by testing the fix this points at: this repository's
composer.jsonran itsmutatescript as a barepest --testsuite=Feature --mutate …(novendor/bin/prefix, no@php). Rewriting it to@php vendor/bin/pest --testsuite=Feature --mutate …— Composer's@phpdirective normalizes the separator before PHP runs — eliminated the launch failure entirely: the same 38-mutantAuth.phprun went from 38-of-38 failed launches (scored killed, reported100.00%) to a genuine run with real per-mutant timing and a real (non-fabricated) result.So the fix is available at the call site rather than needing a change in this project, for anyone hitting the same symptom on Windows: make sure the outer
--mutateinvocation's ownargv[0]is backslash-separated (an@php-prefixed Composer script, or an explicitly backslashed path if invokingvendor\bin\pestdirectly) — not a change toCliConfiguration.php, which was never the source. The suggested fix in the original report (normalizing the literal atCliConfiguration.php:64) would not have addressed this, since that literal is never spawned.What might still be worth a framework-level fix, if the maintainers want it:
MutationTest::start()trusts the outer process's ownargv[0]verbatim for every per-mutant child, with no normalization. A caller whose own invocation happens to produce a separator the child can't use (as a bare Windows invocation does) gets a silent, plausible-looking fabricated result rather than an error —hasFinished()maps any non-zero exit, launch failure included, toMutationTestResult::Tested. Normalizingargv[0]'s separator once, inMutationTest::start(), before building the child command, would close this regardless of how the outer process was invoked.[-][mutate] Windows: hardcoded forward-slash 'vendor/bin/pest' fails to launch, every mutant silently reports killed[/-][+][mutate] Windows: forward-slash outer argv[0] fails to launch per-mutant children, every mutant silently reports killed[/+]drpark commentedon Oct 14, 2026
Confirming on another setup, with a fix that works at the plugin level.
Environment: Windows 11, PHP 8.4.15 ZTS (x64, not Herd), Xdebug 3.5.3, Pest 3.8.6,
pestphp/pest-plugin-mutatev3.0.5,symfony/processv7.4.8. Outer command:php vendor/bin/pest ... --mutate.Symptom, same as reported: 2,647 mutants over a Laravel
app/Domain, all scored as killed, 100.00 %, with ~0.25 s per mutant that looked like real work. A test asserting nothing (expect(true)->toBeTrue()) also gave 100 %. Each child failed with'vendor' n'est pas reconnu en tant que commande interne...(French Windows).Fix applied locally (composer patch), in
MutationTest::start():command: [ ...(PHP_OS_FAMILY === 'Windows' ? [PHP_BINARY] : []), ...$originalArguments, '--bail', '--filter="'.implode('|', $filters).'"', ],Running the child through
PHP_BINARYmakes the launch independent of how the outer process was invoked (separator,.batproxy, path with spaces: Symfony Process quotes it). After the fix, the same run reports 78.43 % (571 survivors), and the no-assertion test drops to 0 %.Data point for #1927: with this prefix and
symfony/processv7.4.8, the quoted--filtercontaining|does not break here. A line covered by two tests runs both children correctly (checked with two no-assertion tests: 0 %, no false kill). That may be because the command line now starts with a quoted token, or a difference betweensymfony/process7.4 and 8.1.Two more fixes were needed on this setup to get trustworthy numbers: the drive-letter absolute path one (pestphp/pest-plugin-mutate#36, otherwise
covers()/--classcreate 0 mutations), and a timeout issue filed as #1968 (the 5 s minimum per-mutant timeout is shorter than a child boot under Xdebug, so some untested mutants are scored as killed).