Repair a corrupt (NUL-byte) placeholder SHA at read time - #2081
Open
tyrielv wants to merge 1 commit into
Open
Conversation
tyrielv
force-pushed
the
tyrielv/repair-malformed-sha-placeholder
branch
2 times, most recently
from
August 13, 2026 17:46
49b77d1 to
1fa041e
Compare
tyrielv
force-pushed
the
tyrielv/repair-malformed-sha-placeholder
branch
from
August 18, 2026 20:39
1fa041e to
1536a33
Compare
When a user process reads a virtualized placeholder whose stored content-id is corrupt - 40 NUL bytes instead of a hex blob SHA - GVFS cannot hydrate it from the corrupt content-id. microsoft#2074 makes that read fail cleanly (no crash, no retry storm). This change goes one step further and repairs the underlying data so the file works again. Telemetry shows this corruption is durable and localized: the same 1-4 files per machine fail repeatedly over multiple days until something rewrites the placeholder (~61 machines / ~6.2K events over 30 days). It is old and version-agnostic (spans >=4 GVFS builds), not a 2.0 regression. The triggering processes are readers (git.exe, copilot.exe, Code.exe); the placeholder was already corrupt on disk. The authoritative path->SHA still exists, because the path is still projected and the git index projection can return the correct SHA for it. Read-time self-heal (WindowsFileSystemVirtualizer.GetFileStreamHandlerAsyncHandler): - Plumb virtualPath into the handler and, when the placeholder's decoded SHA is not a valid hex SHA, recover the authoritative SHA for virtualPath from GitIndexProjection.GetProjectedFileInfo and hydrate the blob from that instead of the corrupt content-id. - The recovery passes a null BlobSizesConnection: repair needs only the SHA, not the blob size, so size resolution (which can throw SizesUnavailableException) is skipped and a size-lookup fault cannot deny a SHA-only self-heal. - A successful hydration writes the whole file, which converts the placeholder into a full file on disk. The corrupt content-id is superseded and future reads never call back, so the file is repaired for good. - If the path is no longer projected (deleted/renamed), the projection lookup throws, or the recovered SHA cannot be hydrated, fall back to the same clean, non-crashing FileNotAvailable failure as microsoft#2074. We deliberately do NOT rewrite the placeholder's content-id in place via UpdateFileIfNeeded. Confirmed empirically against real inbox ProjFS with a throwaway probe: (1) serving the full content converts the placeholder to a full file, so a second read issues no GetFileData callback (hydration alone is the repair); and (2) UpdateFileIfNeeded on the file mid-read returns 0x80070020 (ERROR_SHARING_VIOLATION) because the reader holds the file open. The same probe showed a corrupt placeholder is only injectable from the owning virtualization instance (WritePlaceholderInfo accepts an all-NUL content-id) and that an external FSCTL_SET_REPARSE_POINT rewrite is blocked (ERROR 1359), so this behavior is covered by unit tests rather than a functional test. Telemetry funnel (paired with microsoft#2074's *_MalformedBlobSha detection): - Repaired: *_MalformedBlobShaRepaired (Warning) with the recovered SHA, so we can watch the corrupt-placeholder population drain. - Repair miss: *_MalformedBlobShaRepairFailed (Warning) tagged with a MalformedShaRepairFailureReason (ProjectionMiss / ProjectionException / HydrateFailed / HydrateException). The failed event is emitted on every repair-failure exit - including hydration failures that throw after a SHA is recovered (size mismatch, local IO, ProjFS write failure) - so that repaired + repair-failed accounts for every repair attempt. Coordinates with microsoft#2071: a repair miss stays telemetry category Unexpected; a successful repair simply succeeds. No new BlobHydrationFailureCategory value. Two known, accepted behaviors are documented in code: the projection is read live, so a concurrent checkout can change the projected SHA between placeholder open and repair (serving the currently-projected SHA is the best answer for an already-corrupt file and matches the placeholder-creation path); and an unrepairable-but-projected file whose blob is unavailable pays the normal download + retry budget per read (the same cost any valid-but-unavailable placeholder pays), bounded and never re-crashing. Stacked on microsoft#2074 (tyrielv/fix-invalid-sha-hydration), which is stacked on microsoft#2071. Targets vnext: this is a new behavioral change on the read path for an old, rare, pre-existing corruption, so it does not belong on the 2.0 stabilization line. microsoft#2074 already removes the crash and retry storm on master. Unit tests (WindowsFileSystemVirtualizerTests) cover: repair success (asserting hydration uses the RECOVERED SHA, not the corrupt one) emits *_MalformedBlobShaRepaired and completes Ok; a non-projected path, a throwing projection lookup, an unhydratable recovered SHA, and a hydration that throws after recovery each emit *_MalformedBlobShaRepairFailed with the expected reason and fail cleanly; mid-repair cancellation emits neither repair event; and a valid content-id still hydrates with no repair telemetry. Assisted-by: Claude Opus 4.8 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
tyrielv
force-pushed
the
tyrielv/repair-malformed-sha-placeholder
branch
from
August 18, 2026 20:53
1536a33 to
99fc993
Compare
tyrielv
marked this pull request as ready for review
August 18, 2026 21:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Stacked on #2074 (which was stacked on #2071). Both have now merged and landed in
vnext, so this branch is rebased ontovnextand the diff below is just the repair.What this does
Repairs a corrupt (NUL-byte) placeholder content-id at read time instead of only failing cleanly.
When a user process reads a virtualized placeholder whose stored content-id is corrupt — 40 NUL bytes instead of a hex blob SHA — GVFS cannot hydrate it from that content-id. #2074 makes the read fail cleanly (no crash, no retry storm). This PR goes one step further and repairs the underlying data so the file works again.
Root cause (NUL-byte, not
AllZeroSha)A placeholder stores its content-id (the 40-char blob SHA) as UTF‑16 in the ProjFS reparse point. A corrupt placeholder decodes back to a content-id of 40 NUL characters (
\^@×40) — literal on-disk corruption. This is notGVFSConstants.AllZeroSha(40 ASCII'0', which is valid hex and yields directory"00").Telemetry evidence
GitIndexProjection.GetProjectedFileInforeturns the correct SHA for it, so the corrupt content-id is recoverable from the path.Design (read-time self-heal)
In
WindowsFileSystemVirtualizer.GetFileStreamHandlerAsyncHandler:virtualPathinto the handler.virtualPathfrom the git index projection (GetProjectedFileInfo, called with a nullBlobSizesConnection— repair needs only the SHA, not the blob size) and hydrate from that instead of the corrupt content-id.FileNotAvailableas Fail blob hydration cleanly on a malformed (NUL-byte) placeholder SHA #2074.Why not rewrite the content-id in place?
We deliberately do not call
UpdateFileIfNeededto rewrite the content-id. Confirmed empirically against real inbox ProjFS with a throwaway probe:GetFileDatacallback — hydration alone is the repair.UpdateFileIfNeededon the file mid-read returns0x80070020(ERROR_SHARING_VIOLATION) because the reader holds the file open.WritePlaceholderInfoaccepts an all-NUL content-id); an externalFSCTL_SET_REPARSE_POINTrewrite is blocked (ERROR 1359).That last point is why this behavior is covered by unit tests rather than a functional test — the corruption cannot be injected into a real mount from outside the provider.
Telemetry funnel
Paired with #2074's
*_MalformedBlobShadetection. The two events together account for every non-aborted repair attempt (a read cancelled mid-repair, or aborted because the app closed the handle, emits neither — it is retried, not a repair outcome), so repaired + repair-failed is a reliable rollout signal:GetFileStreamHandlerAsyncHandler_MalformedBlobShaRepaired(Warning) with the recovered SHA — lets us watch the corrupt-placeholder population drain.GetFileStreamHandlerAsyncHandler_MalformedBlobShaRepairFailed(Warning) tagged with aRepairFailedReason(ProjectionMiss/ProjectionException/HydrateFailed/HydrateException), then a clean fail.Relationship to the stack
tyrielv/fix-invalid-sha-hydration), itself stacked on Split blob-hydration and directory-enumeration failure telemetry by cause #2071. Fail blob hydration cleanly on a malformed (NUL-byte) placeholder SHA #2074 is the graceful-fail fallback for every repair-miss branch here.Unexpected; a successful repair simply succeeds. No newBlobHydrationFailureCategoryvalue.Why vnext, not master (2.0)
Per the branch model: master is the shippable/stabilization line; vnext is next-train behavioral work. This repair is a new behavioral change on the read path (projection lookup, a possible cache-server download, placeholder repair, edge cases like deleted/renamed paths) for an old, rare, pre-existing corruption. Adding it to the 2.0 stabilization line would spend stabilization risk on a rare problem that is not worse in 2.0. #2074 (the crash / graceful-fail fix) already removes the crash and retry storm and correctly targets master.
Tests
WindowsFileSystemVirtualizerTests(unit) — the repair success/miss branches and the full telemetry funnel:*_MalformedBlobShaRepaired, completesOk.*_MalformedBlobShaRepairFailedreasonProjectionMiss, fails cleanly.ProjectionException, fails cleanly.HydrateFailed, fails cleanly.GetFileStreamException; and a generic exception) → reasonHydrateException, fails cleanly.Full unit suite: 925 passed, 0 failed (11 pre-existing ignored).