What's wrong
BranchStateHandler (GitBranchStateCache/Endpoints/BranchStateHandler.cs:91) takes context.RequestAborted and passes it to fetcher.EnsureCurrentAsync (lines 156 and 362). That token then goes all the way down to the git process:
MirrorFetcher.EnsureCurrentAsync has the leader run CloneAsync / FetchAsync with the leader request's own cancellation token.
GitRunner.RunAsync (Git/GitRunner.cs:84-91) kills the git process tree and rethrows OperationCanceledException when that token fires.
- The leader's ticket is disposed without
Complete(true), so FollowAsync (Mirrors/MirrorFetcher.cs:~100-120) sees a failed leader. With no mirror on disk yet, it returns Unavailable("…the request that was creating one did not finish.") and does not try again.
So one client hanging up cancels shared work that every coalesced request is waiting on.
Failure scenario
- Request A (leader) starts
git clone for a repository that isn't mirrored yet. The repository is large, so the clone takes longer than A's client HTTP timeout.
- Request B (follower) arrives for the same repository and waits on the flight.
- A's client times out and disconnects. The clone is killed and the staging directory is discarded.
- B's client is still connected, but it gets
503 no-mirror.
- The next request becomes leader, starts the clone from zero, and hits the same timeout.
For any repository whose clone takes longer than the clients' timeout, the mirror is never created. The same thing on a fetch hands followers Stale instead of Current.
This was reproduced with a scratch test using a fake runner that behaves like GitRunner on cancellation (the existing GitRunnerTests.RunAsync_WhenTheRequestIsAbandoned_KillsTheTreeAndPropagatesTheCancellation pins that behaviour):
leader: cancelled (client went away)
follower (still connected, never cancelled): Unavailable / No mirror exists for this repository and the request that was creating one did not finish.
clone invocations: 1
Suggested fix
- Run clone and fetch under a token owned by the service, linked to
IHostApplicationLifetime.ApplicationStopping plus FetchTimeout, instead of the request's token.
- Leader and followers each await the shared task with their own request token (
.WaitAsync(requestToken)). A disconnecting caller then stops waiting, but the work continues.
- Or, as a minimum, when the leader was only cancelled (not failed), let a still-connected follower take over as leader instead of returning
Unavailable.
Acceptance criteria
- A test where the leader's token is cancelled mid-clone while a follower waits ends with the clone completing and the follower getting
Current.
- The git process is still killed on host shutdown and after
FetchTimeout.
What's wrong
BranchStateHandler(GitBranchStateCache/Endpoints/BranchStateHandler.cs:91) takescontext.RequestAbortedand passes it tofetcher.EnsureCurrentAsync(lines 156 and 362). That token then goes all the way down to the git process:MirrorFetcher.EnsureCurrentAsynchas the leader runCloneAsync/FetchAsyncwith the leader request's own cancellation token.GitRunner.RunAsync(Git/GitRunner.cs:84-91) kills the git process tree and rethrowsOperationCanceledExceptionwhen that token fires.Complete(true), soFollowAsync(Mirrors/MirrorFetcher.cs:~100-120) sees a failed leader. With no mirror on disk yet, it returnsUnavailable("…the request that was creating one did not finish.")and does not try again.So one client hanging up cancels shared work that every coalesced request is waiting on.
Failure scenario
git clonefor a repository that isn't mirrored yet. The repository is large, so the clone takes longer than A's client HTTP timeout.503 no-mirror.For any repository whose clone takes longer than the clients' timeout, the mirror is never created. The same thing on a fetch hands followers
Staleinstead ofCurrent.This was reproduced with a scratch test using a fake runner that behaves like
GitRunneron cancellation (the existingGitRunnerTests.RunAsync_WhenTheRequestIsAbandoned_KillsTheTreeAndPropagatesTheCancellationpins that behaviour):Suggested fix
IHostApplicationLifetime.ApplicationStoppingplusFetchTimeout, instead of the request's token..WaitAsync(requestToken)). A disconnecting caller then stops waiting, but the work continues.Unavailable.Acceptance criteria
Current.FetchTimeout.