What's wrong
GitRunner.RunAsync (GitBranchStateCache/Git/GitRunner.cs) starts both reads with a strict UTF-8 decoder:
Task<string> standardOutput = process.StandardOutput.ReadToEndAsync(CancellationToken.None);
Task<string> standardError = process.StandardError.ReadToEndAsync(CancellationToken.None);
...
await process.WaitForExitAsync(linked.Token);
When git writes bytes that aren't valid UTF-8, ReadToEndAsync throws DecoderFallbackException and the task faults. After that, nothing reads the pipe. The fault is only observed after the process exits, but if git still has more than one pipe buffer (about 64 KB) to write, git blocks on the full pipe and never exits. WaitForExitAsync then runs until invocation.Timeout, the process is killed, and the catch (OperationCanceledException) branch returns "The git command exceeded its timeout.". The catch (DecoderFallbackException) branch, which was written for exactly this case, is never reached.
Why it matters
diff-tree on a branch with one non-UTF-8 path (for example an asset named with Latin-1 bytes) plus more than about 64 KB of other changes:
- Each request stalls for the whole
DiffTimeout (2 minutes) and returns DiffOutcome.Timeout() rather than a clear "cannot be read" failure.
- Failures aren't cached, so every heartbeat repeats the stall.
DiffCache followers wait DiffTimeout and then recompute, which stalls again.- The operator sees what looks like a slow git rather than a data problem.
- A single ref name with bytes ≥ 0x80 (git allows these) has the same root cause and a second symptom:
ls-remote --heads and for-each-ref return the UTF-8 failure even when their output is small.- The
AdmissionGate probe never looks at the probe's output, only whether it succeeded, yet the probe fails. So every caller of that repository gets not-admitted, or a timeout (504) if more than 64 KB of refs follow the bad one.
RefResolver returns null, and /branches and /state return refs-unreadable.
Reproduction
I built a repository with a path b\xff.uasset plus 1,500 further files (219 KB of -z output). I then called the real GitRunner with a 10 s timeout:
diff-tree: exit=-1 timedOut=True ok=False elapsed=10.0s summary='The git command exceeded its timeout.'
With the bad path and only a few other files, the call returns immediately with the intended not valid UTF-8 summary. The difference is purely whether git outlives a full pipe buffer.
Suggested fix
- Read stdout and stderr as raw bytes (
process.StandardOutput.BaseStream.CopyToAsync(memoryStream)) and decode strictly only after the process has exited. The pipes are always drained, git exits promptly, and the DecoderFallbackException branch reports the real cause. Alternatively, observe a faulted read task and kill the process immediately, but draining bytes is simpler and cannot deadlock.
- Separately, the admission probe (
ls-remote) only needs the exit code. It shouldn't decode its output at all, so an unusual ref name can't deny admission to a whole repository.
Acceptance criteria
- A
diff-tree whose output contains invalid UTF-8 and is larger than the pipe buffer returns the "not valid UTF-8" result well inside DiffTimeout, with TimedOut == false.
- A regression test covers large output with a non-UTF-8 path, for example a fake git script that writes
\xff followed by 200 KB.
- Admission succeeds for a reachable repository that has a non-UTF-8 branch name.
What's wrong
GitRunner.RunAsync(GitBranchStateCache/Git/GitRunner.cs) starts both reads with a strict UTF-8 decoder:When git writes bytes that aren't valid UTF-8,
ReadToEndAsyncthrowsDecoderFallbackExceptionand the task faults. After that, nothing reads the pipe. The fault is only observed after the process exits, but if git still has more than one pipe buffer (about 64 KB) to write, git blocks on the full pipe and never exits.WaitForExitAsyncthen runs untilinvocation.Timeout, the process is killed, and thecatch (OperationCanceledException)branch returns"The git command exceeded its timeout.". Thecatch (DecoderFallbackException)branch, which was written for exactly this case, is never reached.Why it matters
diff-treeon a branch with one non-UTF-8 path (for example an asset named with Latin-1 bytes) plus more than about 64 KB of other changes:DiffTimeout(2 minutes) and returnsDiffOutcome.Timeout()rather than a clear "cannot be read" failure.DiffCachefollowers waitDiffTimeoutand then recompute, which stalls again.ls-remote --headsandfor-each-refreturn the UTF-8 failure even when their output is small.AdmissionGateprobe never looks at the probe's output, only whether it succeeded, yet the probe fails. So every caller of that repository getsnot-admitted, or a timeout (504) if more than 64 KB of refs follow the bad one.RefResolverreturns null, and/branchesand/statereturnrefs-unreadable.Reproduction
I built a repository with a path
b\xff.uassetplus 1,500 further files (219 KB of-zoutput). I then called the realGitRunnerwith a 10 s timeout:With the bad path and only a few other files, the call returns immediately with the intended
not valid UTF-8summary. The difference is purely whether git outlives a full pipe buffer.Suggested fix
process.StandardOutput.BaseStream.CopyToAsync(memoryStream)) and decode strictly only after the process has exited. The pipes are always drained, git exits promptly, and theDecoderFallbackExceptionbranch reports the real cause. Alternatively, observe a faulted read task and kill the process immediately, but draining bytes is simpler and cannot deadlock.ls-remote) only needs the exit code. It shouldn't decode its output at all, so an unusual ref name can't deny admission to a whole repository.Acceptance criteria
diff-treewhose output contains invalid UTF-8 and is larger than the pipe buffer returns the "not valid UTF-8" result well insideDiffTimeout, withTimedOut == false.\xfffollowed by 200 KB.